HN Explorer

Show HN: OpenMetadata – OSS platform for data discovery observability governance (open-metadata.org)

19 points by jpnguyen ↗ HN
Hi HN,

OpenMetadata is an open-source metadata platform built to solve complex data management challenges and deliver high quality data assets. It’s based on the experiences of solving the data problems encountered at companies like Uber and Hortonworks. Our team includes the creators of Hadoop, Atlas, and Uber Databook, as well as committers/PMC of projects like Apache Kafka and Storm.

Unified Metadata Platform

* Unified: A single platform for data discovery, governance, and data observability.

* API-first: Standardized metadata specifications using well-designed REST APIs, crucial for anyone consuming or pushing metadata for modern data teams.

* Metadata platform: The data catalog is just one of the many applications of metadata built on the platform, vs different siloed data catalog tools, quality tools, governance tools, etc. that each have to be set up and maintained.

Please check out more details about our project at https://open-metadata.org and join our community in Slack https://slack.open-metadata.org. You can also try out our live product sandbox with demo data at https://sandbox.open-metadata.org/.

12 comments

[ 3.0 ms ] story [ 36.6 ms ] thread
DavidMankin
How does this compare to DataHub? I have been comparing the two and I don't have any pros/cons that really help me pick. What I've noticed:

* It seems like DataHub has an async Kafka ingestion approach while OpenMetadata is API

* It seems that the data model of OpenMetadata can be extended/modified by API calls while DataHub only allows for changes to be made in code (in a forked version, no less) and deployed.

* It seems that DataHub comes with more data types out-of-the-box such as JSON Schema

harsha_ch
* It seems like DataHub has an async Kafka ingestion approach while OpenMetadata is API

We do not use Kafka by default. If someone needs kafka they can add it. However for Metadata APIs, we do not feel like Kafka is needed. Lot of projects are getting dependent on Kafka and calling it as real-time. Its unnecessary burden on users who are going to operate in production for 99% of use-cases Kafka is not needed, coming from a Kafka committer :)

2. Yes all of our APIs and Entity definitions are generated using JsonSchema. For us, Json Schema has been awesome, all of our backend / ingestion and UI is generated from JsonSchema and its easy to extend and add new models when needed

3. IMO, we have much more coverage , you can look at the types available here https://github.com/open-metadata/OpenMetadata/tree/main/open... and we are support JsonSchema as a type from a long time

chirag-madlani
Indeed fastest growing OSS project. I love the support and active slack community.
[deleted]
(comment deleted)
shah_sank3t
It is one of the fastest-growing OSS projects focusing on a great user experience. Not all open-source projects focus on UI, as the OpenMetadata team does.
bloody_user
Fastest growing OSS project in data cataloging space! Love the Slack community support and the business challenges the project aims to solve for Organisations!
sushi30
wish I knew about this project sooner. so much easier to manage our data ecosystem with OpenMetadata. We used to have everything scattered all over the place. catalog here, docs on a siloed website, conversations in slack and for data quality we had a legacy process that was only accessible to engineers. With OpenMetadata everything is one place which means better collaboration and faster deliveries!
aballiet
Fast growing project with a supportive community. A lot of connectors available and an UI that is smooth. Deployment is easy with a lot of possibilities to bundle with managed services (elastic, Postgres, external orchestrate like Kestra in my case).

Easy to build on top of it thanks a large range of APIs endpoints making it easy to query what's needed.

pmbrull
+ a lot of great contributors such as yourself :)

Glad to have you in the community!