There seem to be a lot of these, how does it compare to Helix DB for example? Also, why would you ever want to query a database with GraphQL, for which it was explicitly not made for that purpose?
I really wish people would stop using the language as an argument and that commenter would also move on to a more interesting debate.
In your discussion the first comment from an ex kuzu dev made an excellent point that rust for databases in an excellent language to ship faster with confidence while reducing real problems of concurrency and corruption.
At some point it becomes intellectual dishonesty to dismiss a language because of vibes instead of merit.
I didn't dismiss the language. I called it a north star. Rust is still the best option if you desire memory safety.
But rewriting a complex working piece of software in Rust is not trivial. Having an incremental path (where only parts are rewritten in Rust and compatible with C++ code) would be a good path to get there.
Also open to new code and extensions getting written in Rust.
Does anyone have any experience with this DB? Or context about where it came from?
From the commit history it's obvious that this is an AI coded project. It was started a few months ago, 99% of commits are from 1 contributor, and that 1 contributor has some times committed 100,000 lines of code per week. (EDIT: 200,000 lines of code in the first week)
I'm not anti-LLM, but I've done enough AI coding to know that one person submitting 100,000 lines of code a week is not doing deep thought and review on the AI output. I also know from experience that letting AI code the majority of a complex project leads to something very fragile, overly complicated, and not well thought out. I've been burned enough times by investigating projects that turned out to be AI slop with polished landing pages. In some cases the claimed benchmarks were improperly run or just hallucinated by the AI.
So is anyone actually using this? Or is this someone's personal experiment in building a resume portfolio project by letting AI run against a problem for a few months?
Hi, I'm the one building grafeo, I have no idea why it is being posted everywhere. But I can probably answer your questions.
The first version was largely a (slightly rearchitected) port of a local graph database I had been building called graphos. Most of the engine and core are handwritten, so are the python bindings and conformance tests. The rest is indeed largely AI generated, so is the documentation (Mkdocs). The AI generated parts are curated and validated, although it's not up to par for a production release yet.
This is not a resume portfolio project and in no way related to my day job. I started writing grafeo(then graphos) out of frustration with Neo4j and being inspired by some discussions about database internals with Hännes from duckdb at a conference. I tried ladybug, but found memory usage insanely high and was sure I could do better. Anyone looking for an embedded battle tested graph database should probably still look at ladybug though. Grafeo is not that mature yet.
And to be honest I also have no real plans with grafeo, I am using it myself for now and am very happy with it, but that's n=1. It's fully free and open source and contributors are very welcome, but its also not yet fully where I would want it to be, hence the beta status. I have no commercial interest, but had a lot of fun pouring multiple hundreds of hours in and creating something that I enjoy using myself.
Thank you for the shout out! I looked into your benchmark setup a bit. Two things going on:
- Ladybug by default allocates 80% of the physical memory to the buffer pool. You can limit it. This wasn't the main reason.
- Much of the RSS is in ladybug native memory connected to the python connection object. I noticed that you keep the connection open between benchmark runs. For whatever reason, python is not able to garbage collect the memory.
We ran into similar lifetime issues with golang and nodejs bindings as well. Many race conditions where the garbage collector releases memory while another thread still has a reference to native memory. We now require that the connection be closed for the memory to be released.
Serious question: are there any actually good and useful graph databases that people would trust in production at reasonable scale and are available as a vendor or as open source? eg. not Meta's TAO
plenty of those - I've had to work with dozens of different graph databases integrating them on https://gdotv.com, save for maybe 1-2 exceptions in the list of supported databases on our website, they're all production ready and either backed by a vendor or open-source (or sometimes both, e.g. Apache AGE for Azure PostgreSQL).
There are some technologies that have been around for a long time but really flying under the radar, despite being used a lot in enterprise (e.g. JanusGraph).
I run the development of TypeDB, which doesnt use Cypher but works really well as a graph database. Certainly it, and other graph databases like neo4j, are used in production at scale. However, a lot of oss databases are open core on some level, it just depends on where they draw the line. We draw it at clustering/high availability for the time being, the rest is in the CE version.
I just spent an hour with Grafeo, trying to also get the associated library grafeo_langchain working with a local Ollama model. Mixed results. I really like the Python Kuzu graph database, still use it even though the developers no longer support it.
Ever try https://gdotv.com with it? Really interesting to see folks still using Kuzu despite the archival status. We decided to maintain support for that reason, it's been left in a fairly stable rate which is fantastic. Might be worth checking out LadybugDB (the main fork), migration is pretty easy.
Speaking of embeddable, we just announced cypher syntax for gfql, so the first OSS CPU/GPU cypher query engine you can use on dataframes
Typically used with scaleout DBs like databricks & splunk for analytical apps: security/fraud/event/social data analysis pipelines, ML+AI embedding & enrichment pipelines, etc. We originally built it for the compute-tier gap here to help Graphistry users making embeddable interactive GPU graph viz apps and dashboards and not wanting to add an external graph DB phase into their interactive analytics flows.
We took a multilayer approach to the GPU & vectorization acceleration, including a more parallelism-friendly core algorithm. This makes fancy features pay-as-you-go vs dragging everything down as in most columnar engines that are appearing. Our vectorized core conforms to over half of TCK already, and we are working to add trickier bits on different layers now that flow is established.
The core GFQL engine has been in production for a year or two now with a lot of analyst teams around the world (NATO, banks, US gov, ...) because it is part of Graphistry. The open-source cypher support is us starting to make it easy for others to directly use as well, including LLMs :)
I wonder if people are using (or intend to use) vibe-coded projects like the one linked.
I mean - I understand, some people have fun looking at new tech no matter the source, but my question is is there a person who would be designated to pick a GraphQL in language and would ignore all the LLM flags and put it in production.
I added a super cheap and bad embedding database in a project that allows the agent to call a tool for searching all the content it's built, it seems to work pretty well! This way the agent doesn't need to call a bunch of list tools (which I was worried would introduce lost of data to the context), and can find things based on fuzzy search.
It's not clear that graph-bench in "Tested with the LDBC Social Network Benchmark via graph-bench" is a benchmark that you made. It seems more robust and reliable than "we built a db and a benchmark tool, and our benchmark tool says we're the best". Just a thing to be careful about. You should just state that it's your tool and you welcome feedback to help make it so that other projects being compared are compared in their best light. Something like that might help, I don't know though it's a hard problem.
Every time I look at graph databases, I just cannot figure out what problem they're solving. Particularly in an LLM based world.
Don't get me wrong, graphs have interesting properties and there's something intriguing out these dynamic, open ended queries. But, what features/products/customer journeys are people building with a graph DB.
Every time I explore, I end up back at "yea, but a standard DB will do 90% of this as a 10% of the effort".
Been looking for something like this for a side project. The embedded mode with no external deps is the killer feature for me, hate dragging in a server just to do graph traversal. Going to give it a shot.
Weird project, it's definitely AI assisted, high LoC, but when you see the commits it doesnt look like the average AI slob, and the design is definitely not conventional.
JS tests seem fully AI generated thought.. And big difference in quality between some of the ecosystem repo's. Server, Web and memory all seem very well developed, llamaindex and langchain lower effort.
I think the main thing this project needs is more maintainers, but looking purely at the features of this database, and the fact that it's Apache2-0, make it interesting, at least for me.
38 comments
[ 3.2 ms ] story [ 67.2 ms ] threadWriting it in Rust gets visibility because of the popularity of the language on HN.
Here's why we are not doing it for LadybugDB.
Would love to explore a more gradual/incremental path.
Also focusing on just one query language: strongly typed cypher.
https://github.com/LadybugDB/ladybug/discussions/141
In your discussion the first comment from an ex kuzu dev made an excellent point that rust for databases in an excellent language to ship faster with confidence while reducing real problems of concurrency and corruption.
At some point it becomes intellectual dishonesty to dismiss a language because of vibes instead of merit.
But rewriting a complex working piece of software in Rust is not trivial. Having an incremental path (where only parts are rewritten in Rust and compatible with C++ code) would be a good path to get there.
Also open to new code and extensions getting written in Rust.
From the commit history it's obvious that this is an AI coded project. It was started a few months ago, 99% of commits are from 1 contributor, and that 1 contributor has some times committed 100,000 lines of code per week. (EDIT: 200,000 lines of code in the first week)
I'm not anti-LLM, but I've done enough AI coding to know that one person submitting 100,000 lines of code a week is not doing deep thought and review on the AI output. I also know from experience that letting AI code the majority of a complex project leads to something very fragile, overly complicated, and not well thought out. I've been burned enough times by investigating projects that turned out to be AI slop with polished landing pages. In some cases the claimed benchmarks were improperly run or just hallucinated by the AI.
So is anyone actually using this? Or is this someone's personal experiment in building a resume portfolio project by letting AI run against a problem for a few months?
The first version was largely a (slightly rearchitected) port of a local graph database I had been building called graphos. Most of the engine and core are handwritten, so are the python bindings and conformance tests. The rest is indeed largely AI generated, so is the documentation (Mkdocs). The AI generated parts are curated and validated, although it's not up to par for a production release yet.
This is not a resume portfolio project and in no way related to my day job. I started writing grafeo(then graphos) out of frustration with Neo4j and being inspired by some discussions about database internals with Hännes from duckdb at a conference. I tried ladybug, but found memory usage insanely high and was sure I could do better. Anyone looking for an embedded battle tested graph database should probably still look at ladybug though. Grafeo is not that mature yet.
And to be honest I also have no real plans with grafeo, I am using it myself for now and am very happy with it, but that's n=1. It's fully free and open source and contributors are very welcome, but its also not yet fully where I would want it to be, hence the beta status. I have no commercial interest, but had a lot of fun pouring multiple hundreds of hours in and creating something that I enjoy using myself.
Hope that clarifies some things!
- Ladybug by default allocates 80% of the physical memory to the buffer pool. You can limit it. This wasn't the main reason.
- Much of the RSS is in ladybug native memory connected to the python connection object. I noticed that you keep the connection open between benchmark runs. For whatever reason, python is not able to garbage collect the memory.
We ran into similar lifetime issues with golang and nodejs bindings as well. Many race conditions where the garbage collector releases memory while another thread still has a reference to native memory. We now require that the connection be closed for the memory to be released.
Because the latter is really dumb. I don't mind a software written in C, although I personally wouldn't want to write it anymore.
https://github.com/agnesoft/agdb
Ah, yeah, a different query language.
Typically used with scaleout DBs like databricks & splunk for analytical apps: security/fraud/event/social data analysis pipelines, ML+AI embedding & enrichment pipelines, etc. We originally built it for the compute-tier gap here to help Graphistry users making embeddable interactive GPU graph viz apps and dashboards and not wanting to add an external graph DB phase into their interactive analytics flows.
Single GPU can do 1B+ edges/s, no need for a DB install, and can work straight on your dataframes / apache arrow / parquet: https://pygraphistry.readthedocs.io/en/latest/gfql/benchmark...
We took a multilayer approach to the GPU & vectorization acceleration, including a more parallelism-friendly core algorithm. This makes fancy features pay-as-you-go vs dragging everything down as in most columnar engines that are appearing. Our vectorized core conforms to over half of TCK already, and we are working to add trickier bits on different layers now that flow is established.
The core GFQL engine has been in production for a year or two now with a lot of analyst teams around the world (NATO, banks, US gov, ...) because it is part of Graphistry. The open-source cypher support is us starting to make it easy for others to directly use as well, including LLMs :)
I mean - I understand, some people have fun looking at new tech no matter the source, but my question is is there a person who would be designated to pick a GraphQL in language and would ignore all the LLM flags and put it in production.
Don't get me wrong, graphs have interesting properties and there's something intriguing out these dynamic, open ended queries. But, what features/products/customer journeys are people building with a graph DB.
Every time I explore, I end up back at "yea, but a standard DB will do 90% of this as a 10% of the effort".
JS tests seem fully AI generated thought.. And big difference in quality between some of the ecosystem repo's. Server, Web and memory all seem very well developed, llamaindex and langchain lower effort.
I think the main thing this project needs is more maintainers, but looking purely at the features of this database, and the fact that it's Apache2-0, make it interesting, at least for me.
Especially with OLAP queries.