What is the product? It's really hard to figure from the home page, so I looked up on the web and found this[1]
> Today, we are excited to announce that Mars, Seyi, and I have teamed up to start Metaphor Data, a company dedicated to building out the DataHub ecosystem. Our mission at Metaphor Data is to help all organizations better understand and manage their data through the power of the metadata knowledge graph.
So, it's DataHub (hosted I guess) and ... something else? But I'm still not sure what this "ecosystem" is.
Yeah, the post didn't do a good job of explaining what their product actually is / does.
Their About page is similarly vague:
> We are on a mission to empower individuals and companies to change the world through data. This mission started when we created DataHub at LinkedIn. We’re leveraging the learnings from that experience to build a modern, scalable, and cloud-first data catalog and discovery service.
It sure sounds like vaporware from that description. ¯\_(ツ)_/¯
All I could gather from their website is that they catalog schemas across various databases / data warehouses.
It looks like the team is all engineers — they could really use a marketing & comms person…
I still don't fully get what the product is. In the launch, you state that "Metaphor has its root in DataHub [...] However, Metaphor is not a managed version of DataHub". My observation:
- perhaps intentionally, but you really go to great lengths to obfuscate what the product actually does. How is it NOT DataHub? Is it a new implementation of DataHub? Is it extending DataHub somehow? (I would expect the latter, but it's unclear exactly how it extends it, and you never mention exactly how it's different).
- it's very weird to place that important description in "who we are" (I almost skipped it), instead of e.g. "How does Metaphor work".
- lastly, I kinda' feel that this needs to be in the homepage somewhere, not in a blog post somewhere. Not sure who you're targeting with your marketing copy - but if you're targeting engineers who'd deploy a data catalog, this can't be all fluff. Give me something concrete that I can work with: I'm interested in the domain, I want to implement this in my company, but I need to understand what you're offering. Saying that you have a "platform" tells me nothing. Tell me why I should use Metaphor instead of e.g. deploying DataHub myself.
No, it’s saying how important metadata is, going over the components and key features for a platform to manage metadata, and includes some of the history of that category of software at LI.
If you’re at a company with more than 10-20 people doing data science type work, this is an important service.
Why not? I don't think that's an issue if the actual data is already sitting on a third-party cloud in some middle-of-nowhere undisclosed location or being transited through various other cloud services, right?
Depends on the data, but everyone should be treating metadata at the same classification level as the underlying data. Perhaps even moreso, as metadata leaks information about the company, versus leaking data about the user. (Rightly or wrongly, things that harm the company are more likely to be prioritized.)
Big metadata is one of the big upcoming trends in the data space.
Data tooling has gotten a lot better over the last 10 years with Spark + Airflow + Cloud data warehouses. The next step is getting everything to talk to each other with better details.
I'm losing count of all the metadata management systems being open sourced. First there was Wherehows out of LinkedIn [1]. Then there was Amundsen out of Lyft, which is now stemma.io [2]. Then there was Marquez out of WeWork [3]. Now we have DataHub out of LI. It's a little crazy how underserved this problem was several years ago and how over served it's becoming.
One day we will be able to automagically add an alert to a dashboard saying that the database four systems upstream is having a temporary quality issue.
Any tool that gets us in that direction is good in my book.
18 comments
[ 1.4 ms ] story [ 43.2 ms ] thread> Today, we are excited to announce that Mars, Seyi, and I have teamed up to start Metaphor Data, a company dedicated to building out the DataHub ecosystem. Our mission at Metaphor Data is to help all organizations better understand and manage their data through the power of the metadata knowledge graph.
So, it's DataHub (hosted I guess) and ... something else? But I'm still not sure what this "ecosystem" is.
[1] https://pardhugunnam.medium.com/announcing-metaphor-data-c86...
Their About page is similarly vague:
> We are on a mission to empower individuals and companies to change the world through data. This mission started when we created DataHub at LinkedIn. We’re leveraging the learnings from that experience to build a modern, scalable, and cloud-first data catalog and discovery service.
It sure sounds like vaporware from that description. ¯\_(ツ)_/¯
All I could gather from their website is that they catalog schemas across various databases / data warehouses.
It looks like the team is all engineers — they could really use a marketing & comms person…
- perhaps intentionally, but you really go to great lengths to obfuscate what the product actually does. How is it NOT DataHub? Is it a new implementation of DataHub? Is it extending DataHub somehow? (I would expect the latter, but it's unclear exactly how it extends it, and you never mention exactly how it's different).
- it's very weird to place that important description in "who we are" (I almost skipped it), instead of e.g. "How does Metaphor work".
- lastly, I kinda' feel that this needs to be in the homepage somewhere, not in a blog post somewhere. Not sure who you're targeting with your marketing copy - but if you're targeting engineers who'd deploy a data catalog, this can't be all fluff. Give me something concrete that I can work with: I'm interested in the domain, I want to implement this in my company, but I need to understand what you're offering. Saying that you have a "platform" tells me nothing. Tell me why I should use Metaphor instead of e.g. deploying DataHub myself.
If you’re at a company with more than 10-20 people doing data science type work, this is an important service.
Data tooling has gotten a lot better over the last 10 years with Spark + Airflow + Cloud data warehouses. The next step is getting everything to talk to each other with better details.
I use Amundsen and am pretty happy with it.
[1] https://engineering.linkedin.com/blog/2016/03/open-sourcing-...
[2] https://eng.lyft.com/amundsen-lyfts-data-discovery-metadata-...
[3] https://marquezproject.github.io/marquez/
Any tool that gets us in that direction is good in my book.