Show HN: Depoxy – DataOps UI for the AWS data stack (app.depoxy.dev)
During this year we have been building Depoxy, a tool for managing and optimizing data pipelines on AWS.
The main reasons for building Depoxy are:
the current UIs for managing data pipelines are inefficient
optimizing data pipelines has a lot of potential both in CO2 and financial terms
Where is this inefficiency coming from and how much better we can get?
The UI and the tooling around data services are not integrated at the level it is optimal. This is the reason why having an integrated UI improves the workflows by 50% (22s vs 54s). You can check out the video we have about a simple workflow.
How much optimization?
We have 10 years experience with various clients in this space and based on the previous experiences the optimization potential is between 10 - 30% year over year infrastructure cost. This is a lot for many companies, especially nowadays.
Where are we with this product?
This is the first iteration of the product and we just got started. The next iteration is going to be metadata analytics engine for data that is stored on AWS S3. Based on the metadata we can get insight into the efficiency of the data set and provide guidance for optimizing it.
The second next iteration we are planning to implement a SQL parsing engine that provides automatic data lineage so that people can track how data flows through their system.
Detailed blog post with a lot more:
https://www.depoxy.dev/blog/2022-12-07-introducing-depoxy
Let us know what you think. If you find any problems, issues (even security bugs) please let us know. Thank you in advance.
0 comments
[ 2.2 ms ] story [ 26.7 ms ] threadNo comments yet.