Show HN: New CI/CD Platform for Data Engineers (youtube.com)

In the last year I've become interested in how Data Engineering is evolving to become more like Software Engineering, with the adoption of source control, branching/merging, unit testing, modular code etc.

This has been made possible in part by dbt (https://getdbt.io), which is a really nice tool for defining and running data transformations.

I had the idea to build an Open Source CI/CD platform around dbt, to automate the process and create a defined path to production for data changes.

Though we could of course to do this with Jenkins or Github Actions, the thinking is that we can build a tool which avoids a lot of scripting when developing the pipeline, and provides a user experience that is optimised for Data Engineers, for instance exposing the underlying data as it is transformed and tested.

Here is an introductory blog post with more thinking - https://timeflow.systems/blog/introducing-timeflow

I would be interested in any thoughts on the idea and approach.