Hey everyone, Memgraph CTO here. This is a massive milestone for us that took almost 3 years to reach and I would like to thank everyone who helped us get here! If you have any questions or feedback, I’ll be around all day to answer :D
Thank you! So, Memgraph is compatible with Cypher (http://www.opencypher.org) query language and Bolt protocol (https://boltprotocol.org) which means that existing tools and workloads will work. When it comes to the differences, because of our in-memory first approach, low-level implementation and optimizations Memgraph is more suitable for real-time analytical graph workloads. Please let me know if this makes sense or you would like to learn more about specific details. Out of curiosity, do you have an interesting real-time graph use-case?
Memgraph supports more than 80% of openCypher TCK scenarios (https://github.com/opencypher/openCypher/tree/master/tck). For more details about the actual differences, please take a look here https://docs.memgraph.com/memgraph/reference-overview/differ.... The biggest difference is that Memgraph supports loading custom query modules implemented in almost any programming language and compiled into a dynamic library (.so file). Or, running query modules written as Python scripts.
Yes, Memgraph has an embedded Python interpreter so you can run any NetworkX graph algorithm or any other Python code. The code has access to the whole graph, so a bunch of options exists. The entire Python integration is still in the early days, and we have a bunch of plans on how to improve further. What Python libraries do you generally use?
Would Memgraph be a good app to geolocate all of the data involved with a graph database?
Example: geolocate all of the electric distribution networks around the country7globe that interconnect with the values of the graph?
Karim from Memgraph here. Thanks for the questions. Are you referring to geospatial capabilities? If I understand correctly, you would like to add geospatial coordinates to specific elements in the graph (e.g. power stations, power lines etc) and be able to search and query for specific areas (e.g. countries, cities, etc)?
If that's the case, although we don't have direct support for Geospatial data format, you can easily encode it into your graph by setting coordinate as a property on your nodes and edges.
The intended usage isn't that one because Memgraph is built as a server database system. The communication has to go via the binary protocol. But, because Memgraph is implemented in C/C++ and there is the C API for the query modules already in-place, offering an embedded solution is possible. This feature isn't high on the priorities list at the moment. What would be your target platforms? Mobile or maybe something on the IoT side?
Karim from Memgraph here. That's a good question we get really often. As you can imagine we had our hands full for the past few years building a DB from scratch, and so we didn't really have the resources to put together an open-source project the way we imagine it. It's something we'll explore this year to try to come up with the best way to do it.
Good question :D We haven't spent much time trying out RedisGraph, nor we ran any kind of benchmarks at his point, but from what I can see, I would say that the main differences would:
1. Memgraph is a native GraphDB which means that it's specifically engineered to support graph and graph only whereas RedisGraph is a module that runs on top of key-value store (nothing wrong with that, but it might have some limitation when it comes to performance on complex graph algorithms and traversals)
2. Cypher coverage – We cover about 80% of the cypher query language, where I think RedisGraph covers only a small portion (But I hear that they're improving on that)
Apart from that, I guess you would really need to test both when for specific use-cases to judge performance and scalability. We'll try to provide some benchmarks in the next few weeks.
I hope this helps :D Have you used RedisGraph? Any thoughts?
To answer your first question, the main difference between Memgraph and Dgraph seem to be:
1. The graph model: Property graph for Memgraph vs RDF for DGraph (Depending on the use-case you have one model might be better than the other. E.g ontologies are better served with RDF, whereas graphs with a lot of properties and labels are better served by the property graph model)
2. Memgraph is an in-memory first system where is Dgraph is a disk-based system. Again depending on your use-case and the performance you are looking for, an in-memory system might be better suited.
3. Query language & Ecosystem: we support Cypher and the Bolt Protocol (Same as Neo4j) so we work with a lot of the existing graph tools.
In terms of performance, we don't have official benchmarks yet but we have a few clients that test Memgraph and Dgraph and reported a 3-5x in read performance and about an 8x in write performance for their specific workloads.
Sorry, I missed the second part of you question. You should be able to use the Neo4j GraphQL with Memgraph although we didn't fully test it yet(https://github.com/neo4j-graphql/neo4j-graphql).
I'm community support at Dgraph. Reading that question. I come to clarify some points.
First of all, I don't particularly know Memgraph internally. So, I'll stick to Dgraph and related only.
About Karimtr's reply
1. The graph model: Although Dgraph uses RDF as input data. Dgraph is not a Triple Store per se - and the RDF we have is a customized version, which means that it is not 100% compatible with any RDF model (e.g. Turtle RDF) - But eventually, many RDF syntaxes may be compatible.
The decision to use RDF was made a long time ago, for reasons that I am particularly unaware of. It was long before I joined Dgraph. We also accept JSON as data input. By the way, I also don't understand why Neo4j uses CSV as input data since it is not a Graph standard. RDF itself would be more acceptable than CSV. I assume they use CSV for strategic reasons.
1.1 In practice Dgraph is technically a "Property graph" like. There are no fundamental differences between the Dgraph's graph model with Neo4j other than the language itself and the way the data is stored and injected.
1.2 In Dgraph the data is stored in KV using BadgerDB.
1.3 Ontologies can be represented in any GraphDB. The difference is that Triple Store DBs have a language created to infer data specifically with the concept of ontology. And Triple Stores has standardized data input for this.
2. That's right. However, I think we will soon have the option to keep it in memory. But I don't particularly know how useful this is. Today you can keep some Memory first data, but with the guarantee that they will be saved on disks. This helps in performance when there is no use of NVMe.
3. We have GraphQL+- which is a rich language and inspired by GraphQL. And we also have GraphQL which is an "API" language that is now native in Dgraph. A friendly front-end language. And it works "out of the box" once you mount your schema. Dgraph creates a CRUD model based on your Schema. This reduces production time for your application and less logic on your business side. We are still adding "Black Magic" so that the experience in producing APPs is exceptional. And less code typing.
thanks for joining the conversation and clarifying things. I'm far from being an expert on Dgraph so I learned a few things from your answer.
Regarding the data model, that makes sense. I read that Dgraph support properties so I thought you built something along the lines of the RDF+ framework with seems to support properties and labels.
Yes, you're right, ontologies can be represented in any graph but it's clear that RDF is a better option due to the reasons you mentioned and other ones. That's the nice thing about the emergence of different graph databases, you get to pick the right system for the right job :)
When it comes to performance, to be honest, I'm not a big fan of random benchmarks, as it's always tricky to get them right, especially for systems that have big differences. We usually let developers do that for their specific use-cases and we just try to help them set up Memgraph as best as possible.
Cheers.
I really like dgraph. Even amidst some rapid development for graphql integration, dev team has been quick to respond to issues. Was happy to see the continued Jepsen testing.
Now that the frontend devs have gotten their attention, I hope dgraph plans to give the python community some love as well by improving the client API and asyncio support and even full integration with networkx.
Yay! Thanks for the love :) :). Did you try filing an issue here https://github.com/dgraph-io/dgraph with specific features you would like to see? We will try to prioritize if we see the traction from a lot of users.
All editions of Memgraph persist data to the disk via write-ahead logging (WAL) and periodic snapshots for log compaction so even though Memgraph is designed to be in-memory first, data is always backed up to disk. If you enable asynchronous (periodic fsync) WAL, you can trade off a small window of durability for better performance.
Before 1.0, Memgraph leveraged the RAFT consensus algorithm for HA which worked great but had some performance implications. Based on feedback from our users and customers, we have decided to switch our HA implementation to the streaming replication model (similar to PostgreSQL) with automatic failover.
Interesting. Would you mind to go into details about dropping RAFT? It would make a great tech blog post imho, but maybe you could share some high level insights on that?
Related to that: have you considered extensions to the RAFT standard e.g. pipelining?
Thank you! We are here to do our fair share and provide the community with the right tools to solve different kinds of graph problems. Exciting space nevertheless and growing fast!
Thank you! Memgraph has built-in BFS, DFS, and weighted-shortest path. These can leverage the query planning (use the details about data distribution to perform better). Since query modules are introduced, algorithms like Page Rank could be easily implemented. Memgraph offers a couple of them out of the box. Last but not least, all algorithms from the Python eco-system could be run inside Memgraph.
That cloud pricing page is totally useless because it provides zero detail on what you get for your money, how much you need for a given use case (M8? M32?), and forces you to calculate monthly cost yourself. Further, without any autoscaling/Serverless option, we wind up paying for, polluting for, and wasting energy for 24/7 instances at fixed capacity.
Cool product, but the deal is unclear and inflexible, so DynamoDB or Postgres are still better options
When is someone going to provide an actual serverless graph db?
It’s unbelievable how much masturbatory self-congratulation goes on in the graph database community while Serverless Postgres and Dynamo are exponentially cheaper and better. Is Cypher really worth a 100x-1000x price hike?
"An integrated ecosystem that would allow data scientists to easily leverage existing data science and machine learning tools to build graph-powered applications with minimum friction."
Graph databases have as much to do with data science as they do with any software applications.
Going to guess a VC made them put that there.
Definitely excited to try it out for a project I'm working on with DAG optimizations.
Yea, we're still discussing the exact roadmap on this side. In the short term, the plan is to integrate better with the whole ecosystem. We are still exploring options on the graph NN side. Do you have any specific algorithms to point out? :D
47 comments
[ 3.3 ms ] story [ 81.7 ms ] threadIf that's the case, although we don't have direct support for Geospatial data format, you can easily encode it into your graph by setting coordinate as a property on your nodes and edges.
Does this answer your question?
1. Memgraph is a native GraphDB which means that it's specifically engineered to support graph and graph only whereas RedisGraph is a module that runs on top of key-value store (nothing wrong with that, but it might have some limitation when it comes to performance on complex graph algorithms and traversals)
2. Cypher coverage – We cover about 80% of the cypher query language, where I think RedisGraph covers only a small portion (But I hear that they're improving on that)
Apart from that, I guess you would really need to test both when for specific use-cases to judge performance and scalability. We'll try to provide some benchmarks in the next few weeks.
I hope this helps :D Have you used RedisGraph? Any thoughts?
Also, does it support GraphQL?
1. The graph model: Property graph for Memgraph vs RDF for DGraph (Depending on the use-case you have one model might be better than the other. E.g ontologies are better served with RDF, whereas graphs with a lot of properties and labels are better served by the property graph model)
2. Memgraph is an in-memory first system where is Dgraph is a disk-based system. Again depending on your use-case and the performance you are looking for, an in-memory system might be better suited.
3. Query language & Ecosystem: we support Cypher and the Bolt Protocol (Same as Neo4j) so we work with a lot of the existing graph tools.
In terms of performance, we don't have official benchmarks yet but we have a few clients that test Memgraph and Dgraph and reported a 3-5x in read performance and about an 8x in write performance for their specific workloads.
I hope this answers your question.
I'm community support at Dgraph. Reading that question. I come to clarify some points.
First of all, I don't particularly know Memgraph internally. So, I'll stick to Dgraph and related only.
About Karimtr's reply
1. The graph model: Although Dgraph uses RDF as input data. Dgraph is not a Triple Store per se - and the RDF we have is a customized version, which means that it is not 100% compatible with any RDF model (e.g. Turtle RDF) - But eventually, many RDF syntaxes may be compatible.
The decision to use RDF was made a long time ago, for reasons that I am particularly unaware of. It was long before I joined Dgraph. We also accept JSON as data input. By the way, I also don't understand why Neo4j uses CSV as input data since it is not a Graph standard. RDF itself would be more acceptable than CSV. I assume they use CSV for strategic reasons.
1.1 In practice Dgraph is technically a "Property graph" like. There are no fundamental differences between the Dgraph's graph model with Neo4j other than the language itself and the way the data is stored and injected.
1.2 In Dgraph the data is stored in KV using BadgerDB.
1.3 Ontologies can be represented in any GraphDB. The difference is that Triple Store DBs have a language created to infer data specifically with the concept of ontology. And Triple Stores has standardized data input for this.
2. That's right. However, I think we will soon have the option to keep it in memory. But I don't particularly know how useful this is. Today you can keep some Memory first data, but with the guarantee that they will be saved on disks. This helps in performance when there is no use of NVMe.
3. We have GraphQL+- which is a rich language and inspired by GraphQL. And we also have GraphQL which is an "API" language that is now native in Dgraph. A friendly front-end language. And it works "out of the box" once you mount your schema. Dgraph creates a CRUD model based on your Schema. This reduces production time for your application and less logic on your business side. We are still adding "Black Magic" so that the experience in producing APPs is exceptional. And less code typing.
About performance. I suggest doing a test against ludicrous mode https://discuss.dgraph.io/t/sharing-some-numbers-from-the-lu...
Cheers.
thanks for joining the conversation and clarifying things. I'm far from being an expert on Dgraph so I learned a few things from your answer.
Regarding the data model, that makes sense. I read that Dgraph support properties so I thought you built something along the lines of the RDF+ framework with seems to support properties and labels.
Yes, you're right, ontologies can be represented in any graph but it's clear that RDF is a better option due to the reasons you mentioned and other ones. That's the nice thing about the emergence of different graph databases, you get to pick the right system for the right job :)
When it comes to performance, to be honest, I'm not a big fan of random benchmarks, as it's always tricky to get them right, especially for systems that have big differences. We usually let developers do that for their specific use-cases and we just try to help them set up Memgraph as best as possible. Cheers.
I'm with your docs in hands, need to finish some tasks and I gonna check it out!
Cheers!
Now that the frontend devs have gotten their attention, I hope dgraph plans to give the python community some love as well by improving the client API and asyncio support and even full integration with networkx.
All editions of Memgraph persist data to the disk via write-ahead logging (WAL) and periodic snapshots for log compaction so even though Memgraph is designed to be in-memory first, data is always backed up to disk. If you enable asynchronous (periodic fsync) WAL, you can trade off a small window of durability for better performance.
Before 1.0, Memgraph leveraged the RAFT consensus algorithm for HA which worked great but had some performance implications. Based on feedback from our users and customers, we have decided to switch our HA implementation to the streaming replication model (similar to PostgreSQL) with automatic failover.
Related to that: have you considered extensions to the RAFT standard e.g. pipelining?
I'm excited seeing the progress being made in this space recently (this, redisgraph, etc) - the wider ecosystem is still immature but promising.
Do you have any plans to release a NodeJs Driver?
Cool product, but the deal is unclear and inflexible, so DynamoDB or Postgres are still better options
When is someone going to provide an actual serverless graph db?
It’s unbelievable how much masturbatory self-congratulation goes on in the graph database community while Serverless Postgres and Dynamo are exponentially cheaper and better. Is Cypher really worth a 100x-1000x price hike?
Graph databases have as much to do with data science as they do with any software applications.
Going to guess a VC made them put that there.
Definitely excited to try it out for a project I'm working on with DAG optimizations.
Two questions, please. What about C#? Do you have some API? What about spatial search? Do you have that integrated in API?
Thanks