The frustrating thing with Neo4j is that they have two modes - a fairly neutered open source version, and a ~$35k/node enterprise version, with absolutely nothing inbetween. They're apparently doing a cloudy hosted enterprise version somewhere inbetween but that's some time out and not a self-hostable option.
I hear you. We're working on it. I will note that we have a startup program where you can get Neo4j Enterprise for free if you're below 50 employees and also have several other programs for educational institutions, journalists, etc. And obviously the long term solution here is a DBaaS with grow-as-you-grow pricing. Watch this space! :)
There are a ton of use cases in bigger enterprises (1000+ employees) for smaller applications that are not worth the costs you are offering currently, so I'm glad you're working on it. Cypher is amazing (as is most of Neo4j). I've used Neo4j for many years, going back to pre 2.0 versions. The database has evolved massively since then.
yeah, my experience was the same. It's a really nice database to play with or for a small project but if you need a multi-node HA cluster, the price quickly becomes prohibitive.
I guess if I ever need a graph database again I will probably go for dgraph (although I haven't used it in any production environment) - https://dgraph.io or any other graph database that at least has HA setup without 100k/year bill :)
All of this. It's impossible to run Neo4j out of the box in HA in a sane way. Not to mention, you can't even run a reliable backup of the datastore without shutting down the database entirely... unless you pay for their archaic pricing model for enterprise.
Neo4J has been around so long -- is anyone here using it in production and really happy with it, where it's like really crushing a use-case?
I feel like 90% of the applications in existence can go so far with a regular RDBMS that they never try out Neo4j... I know that's the case with me. Half the time I think I'd just try throwing Agensgraph[0] at the problem instead of jumping to the community version of Neo4J.
I used it heavily in production in 2013, for a dating site. I'm sure it has gotten better since then (or at least I hope so). At the time, it was really just a single node. The issue with graph databases where RDBMS doesn't scale is that the graph can grow quite quickly. I've kept a loose eye on the space since then and I think there is quite a few competitors.
This. It's the reason why the heavy hitting products in this space (MarkLogic, OpenLink Virtuoso, AllegroGraph) are all scale out products. A graph can get HUGE very easily.
Indeed, and Neo4j has scaled out horizontally for reads since 2011.
However, the really hard problem with scaling graphs is scaling writes, i.e. partitioning. In my mind, no one has solved that well today because you can't just slap it on top of a partitioning algorithm that's designed for data without explicit relationships (think documents, key value pairs). In order to gain something other than a checkbox feature and "sharding claims" you have to partition based on the shape of the graph at the time of insertion, but also revise it continuously as the graph evolves over time. That's a non-trivial problem that no one has solved today. (Yes, we're obviously working on it here at Neo4j.)
The good news is that you can get really really far with the replicated horizontal scale out model. "Big RAM is growing faster than big data" as you've probably heard and today there are massive Neo4j deployments in production using our third generation scale out architecture (Raft based, multi clustering, causal consistency).
Last year, the MIT team showed the GraphBLAS/D4M model can achieve 100M inserts per second on a cluster [1], and it's improved since then.
The GraphBLAS [2] standard has been in the works for more than 10 years. It's the culmination of the initial D4M matrix model design by Jeremy Kepner [3] and his team at MIT Lincoln Laboratory Supercomputing Center, which models graphs in the language of linear algebra.
And for the last ~5 years or so the GraphBLAS software model has been designed in collaboration with hardware teams at Intel, NVIDIA, IBM, and the labs to make chips and architectures optimized for these new matrix models and capable of scaling out to exascale [4].
From a query perspective the GraphBLAS model is even better now that GPU and TPU accelerators are populating the data centers since it means you can now run local graph queries and global graph analytics algorithms on the same system and both return results in less than a second.
For an overview of GraphBLAS in the context Heterogeneous High-Performance Computing (HHPC) systems running on NVIDIA GPUs and Intel Xeon Phis, see the 2015 talk Scott McMillan [5] gave at the CMU Software Engineering Institute [6].
GraphBLAS is in RedisGraph now [7] -- it uses the official GraphBLAS C implementation written by Tim Davis [8], who as you know implements the underlying sparse matrix algos used in everything from MATLAB to Google Maps -- see his recent talk:
Adoption of the GraphBLAS standard by hardware chip manufacturers is a sign that the golden-age of graphs is upon us -- the the graph hardware/software model has effectively been solved, and the preceding wave created by AI/deep-learning demand led the way for the emergence of GPU/TPU accelerators in the cloud -- all the forces have aligned.
But for whatever reason, I haven't heard anything about Neo4j in terms of GraphBLAS. What's Neo4j's official position on this and the adoption of the GraphBLAS standard?
I realize it's a big change and would require a big architectural overhaul, but it's been in the works for 10 years, and most of the vendors have been involved for the last 5 years.
Has Neo4j been involved in the GraphBLAS design process and/or are you moving toward adopting the standard?
Yes, but your clustering is just read-only replicas. So you still have the problems that when your graph gets huge and you run out of space to scale each node up you're going to have a bad time. And it also ignores that when you're talking about scaling up to truly monstrous machines with TB of memory the cost is astronomical.
I agree, it's not an easy problem to solve, but saying nobody has solved it isn't accurate either - OpenLink (who I mentioned previously) solved it for their RDF engine. At a previous gig using that across an 8 node cluster we were seeing load rates of hundreds of thousands of edges (or in this case, quads) per second.
Yes I've seen it used in many situations where any other non-graph database would struggle. If you're doing security or fraud detection, geospatial analytics, data lineage, anything social, dynamic relationship modelling etc. then graph works well.
RDBMS has its strengths but it isn't suitable for every use case.
We just did our annual conference GraphConnect in NYC last month. There's a BUNCH of talks online of people who are very successfully and very happily using it in production deployments. Check it out here: https://neo4j.com/graphconnect-2018
Btw, if you've ever stayed at a Marriott (or anyone they now own: Starwood, Hilton, Ritz Carlton) then you've used Neo4j in production (to book that room). If you've ever purchased a flight ticket, then you've used Neo4j in production (over 99% of all fare calculations are done with Neo4j). Etc.
Oh, there's a bunch of stuff in addition to that. I could give it a shot, but would probably mess it up. :) Way better if you hear it from the horse's mouth. Here's Scott Grimes from Marriott at GraphConnect 2016:
I used to work at an OTA (online travel agency, ie Booking.com, Hotels.com). There's all kinds of complications to what seems to be a pretty simple thing: a building with rooms that are either occupied or not. There's multiple rates per room for a single night, then there's something called room rate plans which I won't get into. In other words: it's more complicated that we think (or than it needs to be).
This type of things is based solved with Prolog. The issue is that folks are trying to solve declarative problems using procedural/object oriented/functional programming. Because we store data in databases, we have twisted it into a database problem but it's not.
However complex you think this problem is, it's pretty simple. It's a very old problem, scheduling, well understood and solved. Using the wrong tool makes it a nightmare. Using the right tool makes it ridiculously easy. The solution is not the database, but the language.
I too would like to hear about getting facts from an sql db, and writing back results from prolog - it's hard to see how a prolog text file can/could be used as a source for transactional problems like booking (making sure the answer you find can be committed while still valid).
I suppose one could have a prolog service that came up with suggestions, ordered by preference, and then attempt committing them in order to the actual transactional storage layer. But I don't see how you could avoid re-implementing transactions.
With an rdms you could look for options, and attempt a booking in a transaction, and have a fairly established way to resolve conflicts.
Do you have any reference material on using neo4j for complex pricing - we're looking at using neo4j for this, and some evidence that it's going well in other places will increase the comfort level.
Off the top of my head, I know of two big and complex pricing engine projects in production. One is the Marriott one I referenced in a previous post. Scott Grimes spoke about it at GraphConnect 2016:
https://www.youtube.com/watch?v=wdFYEuWWpzo
The other one is confidential but is in the airline industry.
In general I think that graphs are a very good fit for many problems in analytics (e.g. fraud detection, logistics) and having a database that treats graphs as first-class citizens is a good thing as well.
I've used Neo4j in the past and it seemed to be stable and efficient, not sure about how well it scales though.
Before, TitanDB (now JanusDB) in conjunction with the Tinkerpop stack was probably my favorite graph DB/stack, but not sure how seriously Datastacks (who owns TitanDB now) continues developing it. And JanusDB as a project didn't seem very active to me (I could be wrong of course).
That said, you can construct and handle graphs using relational databases, graph databases give you advantages in that they have (often) better indexing (i.e. O(1) lookups of vertices & adjacent edges) and come with graph querying languages (e.g. Gremlin), which make it much easier to work with graphs compared to SQL (you can use recursive CTEs to walk graphs on the database side as opposed the client side but complex queries are hard/impossible to write like that). I've written a graph DB abstraction layer in the past that also supports SQL backends: https://github.com/7scientists/vortex.
Wow, we had used the same stack. Neo4j - gave up because of horizontal scaling issues, moved to TitanDB and then our startup folded.
Fast forward 3 years and am back in the graph world, this time dealing with money laundering and fraud rings. It stumps me as to why graphs are not used more in the financial world. Financial transactions are no different from social interactions. So concepts such as community detection etc apply to the financial realm as well and in fact we are using such concepts to determine fraud rings. I may get back to neo4j.
Feel free to have a look again and let me know your thoughts. emil@neo4j.com or @emileifrem.
20 of the 25 biggest banks in the world use Neo4j today, and fraud detection (specifically fraud rings) is a really popular use case. (So much so that if you search for "fraud detection" in an incog chrome window, https://neo4j.com/use-cases/fraud-detection/ is on the first page of google).
Btw, we've scaled out horizontally for reads since 2011 but our initial Paxos based implementation was shaky. Since then we've revised it twice and our third generation Raft based architecture is significantly better. If you add multi clustering support + causal consistency and a bunch of other improvements I think our horizontal scalability story is very different today from a few years ago. Lemme know if you try it out and disagree!
Hey if you guys are still considering solutions maybe one of the following derivations of Paxos will help:
EPaxos[0] - flexible/minimal quorums + if you're willing to know ops are commutative you can fast-path, and you can flexibly quorum
WPaxos[1] - flexible/minimal quorums + dynamic partitioning via object stealing for spreading writes around
The main benefit of these is of course getting over raft's weakness of the leader being a SPOF.
If you're willing to go with causal consistency there's also CmRDTs (operation/command Replicated Data Types) which are generalizable to "pure operation based CRDTs"[2] which make the merging problem trivial and should in theory boil things down to replication lag while allowing writes at every node.
We're using neo4j for fraud detection in online gambling and it's very useful for spotting rings of duplicate/related account etc.
One benefit I didn't think of before but has become clear: non-technical people find viewing and manipulating the graphs very easy, and explaining the links is much simpler.
Graph databases have been used for YEARS for fraud detection. There was a company called Metatomix out of Boston that did this back in 2008.... I think they got sold to Ab Initio.
Funny to me how tech is constantly a cycle of folks having no idea about the previous incarnations of their ideas.
There are two kinds of graph databases, property graph and rdf semantic graph. Neo 4J is the former, which I don't think is that useful in primarily only showing some kind of relation between things. The rdf semantic graph approach, however, captures the semantics of the relationship allowing inference and queries not normally possible with rdbms.
It's an easier cultural change in thinking from rdbms to property graph, but also not a huge improvement in terms of what you can do vs an rdbms. Going with a full semantic graph where the relationship is also represented by a node with unlimited relationships, and moving and thinking in hierarchies and inference, is a complete cultural change with an impressive productivity and capability improvement not possible with rdbms. Allegrograph is a good example of a semantic graph which can handle trillions of triples.
Do you have a good example of when an RDF is superior? My looking into RDF's was summarized as "way too complex for little gain" but I admittedly was looking at it through my SQL/Neo4j lens.
I'm working on a project with old school RDF & semantic web, and I tend to concur with you. I often miss my usual RDBMS. Maybe it's the way my data is structured, but any complex display of data quickly becomes a weird maze of async calls.
Before getting into Neo4j (and later joining the company), I'd worked with SQL databases a lot and, at one point, as a DBA. I spent the first few months trying to draw direct analogies between graphs and tables but after a while realised that I needed to "unlearn" a load of what I thought I already knew.
It's now pretty refreshing to be able to look from a higher vantage point at the different types of data store available, and work out which is the right one for each job.
It turns out that the graph is actually the best solution more often than you'd think!
Any time you're working with information, which tends to have hierarchical relationships and classification schemes. And usually a particular type of thing can belong to many different hierarchies, so you can find a type, and then traverse the graph among all hierarchies that type belongs to to find results. So you can easily take something, find it's type, indicate what target type things you want to understand it's related to, and get a result. And this can all be done on the fly. Doing this with the incredible number of joins you would have to use with rdbms would be unfeasible with changing hierarchies.
The other REALLY big advantage, the data model never changes! I literally let the graph, to a certain extent, write the code (that is, code is generated based on graph relationships on the fly)
And finally, you can have everything in the "same" location, you are always working with an "information" lake - no need to have many many different tables.
I appreciate your response, and forgive me for asking the same question, but do you have a more tangible example? I'm particularly interested in understanding more by seeing something like the disaster SQL for traversing hierarchies and comparing it to an RDF implementation.
I'm not sure that this is a necessary distinction.
I don't see anything about a semantic graph database that would prevent it from being built on top of a graph database like Neo4j (and we are doing kind of that at my current company).
I've worked extensively with RDF triple stores of various types in the past.
Not impressed. Too academic, too expensive, bad performance.
SPARQL is one of the worst query languages I've seen. We can thank the W3C for that. Using it for a few weeks makes it clear that it was born in an ivory tower committee rather than engineering experience.
> is anyone here using it in production and really happy with it, where it's like really crushing a use-case?
I've used it in production settings where it was NOT crushing a use case. (A couple small internal HR-related sites that would've been better suited to an RDBMS.) Based on that experience, my impression is that it would take a fairly specific graph-oriented problem to get me seriously thinking about paying the costs associated with something like Neo4J when compared to a traditional RDBMS. (Less mature compared to RDBMS systems, less well understood data model, less common query language, less support from other tooling.... all of which can be quite important to the overall costs of a system.)
Neo4J is used in production at my company. We have had problems with the clustering architecture known as HA. It was superseded by causal clustering in latest Neo4J releases. We are still requalifying our app with this new system, but we are reasonably confident that it will improve things significantly.
The biggest selling point is that business people have no problem discussing the graph data model with us, data engineers. And their OGM layer is easy to use (but it lacks support for multiple inheritance, at least in its Java version).
I have only used Neo4j for one customer project, but I experiment with it myself (I also use RDF datastores) and have used Neo4j in examples for two books I wrote. Good technology, and I am glad they are successfully transitioning from the open source community version to also making money with their enterprise version.
It is still a research thing, but I am starting to see occasional papers on inducing relations in graphs using deep learning. If this proves useful, that should help the growth of graph databases and the use of knowledge graphs.
However, as another commenter ‘hardwarsofton’ said, just using Postgres is very often all I need.
RDBMS is always going to be the default database that works for 90% of use cases. But Neo4J (graph), Cassandra (wide), MongoDB (document) etc aren't addressing the common use cases. It's all about the more complex and exotic data models.
Huge fan of Neo4j and there are many data stores that would benefit from a graph, security data being one. And Cypher is a fantastic query language for graphs and should be the standard.
I had a toy project which housed all its data in Postgres and I needed a bit of graph traversal, really basic stuff. I almost prepared some ETL to load the data into neo4j/redisgraph, but then I found out about recursive CTEs in Postgres.
Granted, they won't let you do nearly as much as some advanced graph algorithms, but the ease with which you can use it in your operational data store is amazing. And with proper indexing, I could do a traversal in hundreds of milliseconds.
Many of us who have worked in the graph database space have been tempted to use the graph abstractions on top of a relational database. It's a reasonable first step. Tables are nodes, join tables are relationships and so forth.
It's also however a bit of a dead end once you go beyond the basics. The costs of joins get worse the deeper you go, and "hundreds of milliseconds" is at least an order of magnitude slower than what Neo4j would do for you.
Once you take that major performance penalty, and then layer it into more complex graph algorithms or analytics, it gets really, really painful quickly. Granted, you might not notice this if you never needed to go further than 2-3 hops in a graph. But once you start working with graphs you're not going to want to stick to such basics.
More technical detail on the difference between a graph abstraction on top of another database, and a native graph database, can be found here:
Well, of course it would be expensive to use tables as an arc.
So say A points to B
you have 3 tables right? Table 1 for A, Table(2) for B and a join table (3) showing that A points to B right? Why would you do that? What's stopping you from having one table that contains A and what A points to? So you only have 2 tables?
What if you have a node that can point to many items, a column can contain a list in postgres, so we can still have one Table containing your node data and a list of items they point to.
I'll concede that graph databases are easier to write query for, most people already struggle with basic SQL, let alone CTE and recursive CTE.
I'm yet to be convinced that a problem can't be reshaped and mapped on a traditional RDMS and yet remain performant.
> What's stopping you from having one table that contains A and what A points to? So you only have 2 tables?
So you can do that. Suppose you denormalize the graph into a single table. Either your denormalizing, and to connect A to many B's you have to duplicate A's data in the table, or you have the constraint that the A -> B link can only have a cardinality of 1.
This would be not a good choice for nodes with many relationships to other things, say for example "Person friended Person". It might work if the cardinality was somewhat capped, like say for example "Customer ordered Product" (a common denormalization).
For a real application, you're not going to have 1 of these. You're going to end up with 20+.
> I'm yet to be convinced that a problem can't be reshaped and mapped on a traditional RDMS and yet remain performant.
Any problem can be re-shaped to any database formalism. For that matter we can re-shape everything we're discussing for a straight K/V store like Redis. The expressive power of a database isn't at issue, because any database can store any dataset. Period.
More relevant questions are about performance and about conceptual fit for the problem. First on performance, have a look at the performance growth graph here:
If you reason from the computer science of how these systems work, this result makes sense.
From the conceptual simplicity standpoint, that's kind of a matter of personal taste and application. Can you do it all with an RDBMS? Sure. But all of these different tech niches exist because sometimes you want more than one kind of tool for the wide variety of jobs you need to accomplish.
I'd argue that it's conceptually simpler to think of your graph as nodes and relationships, rather than to remember each time which node/rel set was denormalized into one table, which node label was split out into its own separate table, what that join table was, how the key naming differed between tables, etc. etc. etc. (Because once you have a non-trivial sized graph, you'll have a lot of these, and maybe you made different decisions at different spots).
The prize if you do remember all of that is that you get to write quite complex SQL to join together the datastructures for non-trivial traversals, because substantial graph use cases that you can answer on the basis of a single denormalized table are going to be rare.
In terms of conceptual simplicity, compare a recursive join SQL query to a cypher snippet like "MATCH (user:User {login:'bob'})-[:KNOWS*..5]->(foaf)". The equivalent SQL is...difficult.
It's just a "use the right tool for the job" situation at its core.
AgensGraph looks great. Especially the ability to execute OpenCypher query or embed it in SQL. If it can be used with citus extension and sharding mechanism we can have distributed graph database which supports ACID.
Also, the official site says it's a commercial product. I wonder how many features are supported in community/opensource/free edition ?
I do, but only in smaller side projects and I'd love to hear from someone who has experience with both in larger settings. Although they work very differently, the query language (Cypher vs. Datalog) have a very similar feel to them. When I tried them it seemed that Datomic would be the more flexible of the 2 (eg. for use cases where you'd want a bit of relational and a bit of graph support), and that Neo4j you'd end up with PostgreSQL next to Neo4j in many practical applications. I have to say though that the query/visualization UI that Neo4j provides is nice and helps with making sure your data is indeed stored in the way you want.
I've been using Neo in production on GrapheneDB with py2neo and Flask for about a year. Love it.
The reason I use a graph is for consistency from my product level business logic to my implementation.
Basically, to solve my problem, I started with a set of english statements, which yielded a grammar, that I described as "objects and morphisms," (things and relationships) then implemented it in a graph - put a front end on it and built a product.
Graphs provide coherence to my problem. Could an RDBMS do this? Yes, but not without a complex intermediate query layer. I think of using a graph as analogous to specifying your problem in terms of a functional language instead of imperatively. The reason to do that is because your product is the result of maintaining consistency of an abstraction, like a DSL or a game, instead of just retrieving stored values, documents and their variations.
It's disruptive to a lot of orgs as well, since there is a lot of sunk cost in RDBMS experience, so I think the applications are all net new projects. I don't foresee anyone migrating to one, but I do see a point where majority of new products use one.
Great comment.
Along similar lines, I can see GraphQL replacing traditional REST APIs for an increasingly large percent of new projects, and reaching a majority.
It's interesting that you mention GraphQL. For me, using Neo4j and GraphQL were similar experiences, in that once I started using them they offered a more intuitive and productive approach over the "standard" tools (RDBMS and REST, respectively).
Once I learned Cypher and some common graph data modeling constructs, I found I could build more complex applications faster using Neo4j, largely because I found the graph model of my project's domain more intuitive and easier to work with than a relational database model.
Similarly, building and using GraphQL APIs has been a huge productivity win once I figured out how to build GraphQL services.
Of course, when used together, Neo4j and GraphQL have some great synergies, since it's graph all the way down ;-)
It's also worth pointing out that Neo4j has some great GraphQL integrations [1].
Oh man did I struggle with Cypher to start. Conceptually I felt like I understood it but 2/3rds of my queries would go out to lunch and never come back.
Lessons learned for me:
- start with very small datasets (mine was 10k nodes/40k edges/5 types and that still sent it to the moon in many cases)
- get to know the query planner right away
Cypher seems to reward a more deliberate and iterative approach to learning it rather than the ol' grip it and rip it. Once I settled down a bit it came together quite nicely.
Amazing, never heard of Graphene before. Just checking them out now. Been dabbling lately in Neo for a couple of projects I'm currently working on and feels like a natural fit.
Very glad to hear you've chosen a solid Python stack for your application :)
We absolutely wouldn't be where we are without our community. From drivers and tooling to the huge amounts of support and goodwill that we see, it's a wonderful ecosystem to be a part of.
This is great news for Neo4j, and for the thousands of organisations that are just starting to realise that they have an unmet need that graph technology can solve.
What we are seeing here is the 'commodification' of graph, a trend that happens in technology in general. Companies that launched ten years ago, on a massive investment with their own proprietary graph technology - I'm talking the likes of Twitter, Facebook and so on - today the same features could be implemented with a fraction of the investment. They'll do this by leveraging Neo4j.
This funding we'll broaden the reach of graph technology, while reducing the overall cost for individual organisations to adopt. Social networks, recommendation engines, fraud detection systems are all now easily within reach. Check out our own free and open-source recommendation engine, which was built on top of Neo4j, for example: https://github.com/graphaware/neo4j-reco.
We live in exciting times. While the commodification of what we call 'graph 1.0' is in progress, what Tesla's head of A.I. Andrej Karpathy brands "Software 2.0", that is the intersection of machine learning and software development is rapidly picking up pace. We're only at the beginning of the hype cycle on this. And guess what? IT is an established fact, is that graph is playing a central role in this transformation process.
We are proud to say that our organisation is at the forefront of using graph technology to derive insight and meaning from unstructured data - we call this GraphAware Hume. We're really excited about this!
As you can see we're are pretty passionate about graph technology, and Neo4j in particular, and in our opinion we're at the beginning of what is going to be a very transformative adoption. If you're thinking about exploring how graph might fit into _your_ organisation, of course feel free to reach out.
People think Neo4J is for things like social, but Facebook's social graph doesn't actually work like Neo4J does – Facebook's load is dashboard shaped and read dominated, so query/storage/writer separation is really important. http://www.dustingetz.com/:datomic-facebook-tao/
This is a little off topic, but there are so many graph databases out there. It's hard to know which one to start playing with..
One would think that Neo4j probably is the most stable one? But it's unclear to me if the full version is open source or not? [1][2]
Has anyone tried several?
Has anyone tried Dgraph? [3]
[1] Community is limited according to Wikipedia
[2] Trying to download enterprise takes me to a "Start a Free 30-Day Trial" page
[3] https://dgraph.io/ ?
We wanted to use Neo4j at my last job (in Mexico). However we found the commercial version was prohibitely expensive, and the free version did not work for real life problems.
So I think this is a place where open source alternatives would have been welcomed.
I quite like it, but I haven't gotten to the 'running in production' part so I haven't experienced what it's like to actually manage or scale it, only its query language and setup.
We're currently using it and it's got some pros/cons, though the cons are a little scary. Jepsen did an evaluation of Dgraph[0] shortly after we started using it and I can confirm seeing a lot of what was noted. Dgraph is _fast_, but there are problems with the underlying data store (their own DB called Badger). Some of those issues have been remedied, but many still exist.
As a graph database, it has some non-typical tradeoffs. You can't easily discern incoming edges and there's no true node deletion. There's a pretty narrow happy-path where the DB works as advertised/expected, but it's just a fairly young DB from an understaffed startup. Probably worth waiting a year or two for the kinks to be ironed out.
Congrats to Neo4j on the raise! I hope it changes the perception of US VCs w.r.t. graph DBs, who are falling behind the dev enthusiasm and readiness for adoption in this field.
As Jepsen report mentioned, it had identified 23 issues, 19 of which were resolved before the report released and another one right after. Dgraph has gone a long way since v1.0 release in terms of production stability. I'd recommend trying out the latest v1.0.9 release or the upcoming v1.0.10.
Dgraph itself is close to being launched in production at a few very big and well-known companies (that we can't mention publicly yet), who moved away from Neo4j to Dgraph. Needless to say, Dgraph's performance and scalability far exceed any other graph DB in the market.
Dgraph is tackling a lot harder problem of doing distributed joins and traversals, while providing distributed ACID transactions, synchronous replication and linearizable reads. The equivalent of Spanner, which can also do efficient joins (something relational DBs suck at, so technically more complex). There's no graph product out there like this or even a single paper which Dgraph is based on, rather we had to do original research to perfect this technology -- which is why it took time to build and stabilize Dgraph.
Badger, the underlying kv DB, itself was never found to have an issue. It is serving several petabytes of data in production use at various companies. We built Jepsen style bank tests for Badger, which run successfully nightly, and there's an open bounty of $1337 for finding any data loss bugs in Badger.
Dgraph is decently staffed (7 engineers) for a seed-stage startup, but we're definitely hiring and planning to grow in SF. No need to wait, this is the right time to run Dgraph in production.
Regarding performance: have you seen Roi Lipman's RedisGraph presentation (https://www.slideshare.net/mobile/RoiLipman/graph-algebra)? GraphBLAS is elegant and very promising. There some benchmarks comparing RedisGraph to Neo4j and others at the end of the slides. I would love to see Dgraph benchmarked against RedisGraph the same way.
For those unfamiliar with the tech can you give numbers to frame your scaling challenges. Performance issues is too broad for me to rationalize in a meaningful way without more specificity.
Neo4J is mainly a one-machine database. For larger clusters, you probably want to look at ArangoDB or Dgraph, for examples. Meaning, as long as your data can be stored in one running machine, you're alright. When wanting to scale horizontally, other solutions were built with that first in mind.
The problem with neo4j is that Cassandra is a brilliant alternative with a superb hosted story - lots of providers have hosted Cassandra which scales to petabytes levels.
And there is a huge scalable graph stack with Cassandra - Datastax Enterprise Graph, Titan,JanusGraph (where Google is involved), Tinkerpop,etc.
The production readiness of neo4j is something I'm still not quite sure about.
It truly shines as an embedded graph db though. I wonder if there is a Blockchain story around neo4j (as a replacement for leveldb) that makes this more interesting.
After all there is a lot of excitement around DAG based blockchain alternatives.
Neo4j, and graph databases in general, are an excellent use case for IoT access management.
Our schema involved taking physical assets/personnel and representing them as different labels: machine, factory, production line, user, usergroup, etc. We then drew complex relationships between different user/groups in the organization and the assets they were responsible for.
At first, we used a relational database, but it soon became difficult to go more granular than simply: user belongs to usergroup, usergroup belongs to client, client has factories, factories have lines, lines have machines.
As many have pointed out here, it's not that you can't do this with non-graph databases, it just requires a more complex query layer. Neo4j allowed us to represent complex business relationships as natural language, and that really helped us as the business scaled.
To everyone @ Neo4j, on behalf of the Graphistry team, big congrats!
This may help non-graph folks understand the community a bit. Neo4j has a bunch of cool bits, and it's been a pleasure watching them bring two specific "aha!" moments to customers. Our tech helps teams build scalable visual workflows that include visual graph, so we're often brought in near the beginning of a graph project, and have repeatedly seen two situations where a DB at Neo4j's quality shines:
1. Performance: A teams starts using their existing data stores -- SQL, Splunk, etc. They'll get quite far. Often, however, they will hit some query that just cannot perform. E.g., for two bank accounts, all paths between them. For different DBs and workloads, these can be different things.
2. Ease: Asking for something like a 360 view around a device, user, patient, account, etc. is hard in sQL - you don't know what column, table, etc. to look at. Or imagine the above shortest-paths query. Cypher makes writing this stuff EASY, so in a world where a lot of people can barely do SQL, that's a superpower.
Neo4j has been broadening by entering the scaleout world, app dev world, and adding multi-modal & ML capabilities, which are all important things and help grow the eco-system. Congrats again!
I used to use OrientDB but moved away due to stability issues (long ago now so hopefully they have that sorted). I also just noticed they’ve been acquired by SAP!
For me graph layouts are conceptually superior vs relational when explaining to non techie users.
Practically however I now stick to Postgres - it’s ‘good enough’ (for what I’m doing) and has a heap of benefits in and of itself.
I looked st Agensgraph but I couldn’t get enough info on it, plus it is a custom version of Postgres (not a plug-in) and i think they recently switched to AGPL which makes it overall less exciting to investigate.
I know there are places where having a real graph db helps but I’ve not personally hit those scenarios yet.
Is "nabbing a Series E" really a sign of something flourishing? The way I view investment rounds is as a letter grade. Once you've reached a "Series F" maybe you've just failed? Has this company made any profit, for instance? Just conjecture.
93 comments
[ 5.1 ms ] story [ 206 ms ] threadI guess if I ever need a graph database again I will probably go for dgraph (although I haven't used it in any production environment) - https://dgraph.io or any other graph database that at least has HA setup without 100k/year bill :)
https://neo4j.com/developer/guide-cloud-deployment/
As others pointed out in other threads, this can be done for free for startups of a certain size (https://neo4j.com/startup-program/), and eval licenses are available (https://neo4j.com/lp/enterprise-cloud/?utm_content=aws-marke...)
I feel like 90% of the applications in existence can go so far with a regular RDBMS that they never try out Neo4j... I know that's the case with me. Half the time I think I'd just try throwing Agensgraph[0] at the problem instead of jumping to the community version of Neo4J.
[0]: https://github.com/bitnine-oss/agensgraph
However, the really hard problem with scaling graphs is scaling writes, i.e. partitioning. In my mind, no one has solved that well today because you can't just slap it on top of a partitioning algorithm that's designed for data without explicit relationships (think documents, key value pairs). In order to gain something other than a checkbox feature and "sharding claims" you have to partition based on the shape of the graph at the time of insertion, but also revise it continuously as the graph evolves over time. That's a non-trivial problem that no one has solved today. (Yes, we're obviously working on it here at Neo4j.)
The good news is that you can get really really far with the replicated horizontal scale out model. "Big RAM is growing faster than big data" as you've probably heard and today there are massive Neo4j deployments in production using our third generation scale out architecture (Raft based, multi clustering, causal consistency).
I don't understand this comment.
Last year, the MIT team showed the GraphBLAS/D4M model can achieve 100M inserts per second on a cluster [1], and it's improved since then.
The GraphBLAS [2] standard has been in the works for more than 10 years. It's the culmination of the initial D4M matrix model design by Jeremy Kepner [3] and his team at MIT Lincoln Laboratory Supercomputing Center, which models graphs in the language of linear algebra.
And for the last ~5 years or so the GraphBLAS software model has been designed in collaboration with hardware teams at Intel, NVIDIA, IBM, and the labs to make chips and architectures optimized for these new matrix models and capable of scaling out to exascale [4].
From a query perspective the GraphBLAS model is even better now that GPU and TPU accelerators are populating the data centers since it means you can now run local graph queries and global graph analytics algorithms on the same system and both return results in less than a second.
* Google Cloud GPUs https://cloud.google.com/gpu/
* Google Cloud TPUs https://cloud.google.com/tpu/
For an overview of GraphBLAS in the context Heterogeneous High-Performance Computing (HHPC) systems running on NVIDIA GPUs and Intel Xeon Phis, see the 2015 talk Scott McMillan [5] gave at the CMU Software Engineering Institute [6].
GraphBLAS is in RedisGraph now [7] -- it uses the official GraphBLAS C implementation written by Tim Davis [8], who as you know implements the underlying sparse matrix algos used in everything from MATLAB to Google Maps -- see his recent talk:
* RedisGraph in the Language of Linear Algebra with GraphBLAS, co-presented by RedisLabs and Tim Davis [video] https://www.youtube.com/watch?v=xnez6tloNSQ
Adoption of the GraphBLAS standard by hardware chip manufacturers is a sign that the golden-age of graphs is upon us -- the the graph hardware/software model has effectively been solved, and the preceding wave created by AI/deep-learning demand led the way for the emergence of GPU/TPU accelerators in the cloud -- all the forces have aligned.
But for whatever reason, I haven't heard anything about Neo4j in terms of GraphBLAS. What's Neo4j's official position on this and the adoption of the GraphBLAS standard?
I realize it's a big change and would require a big architectural overhaul, but it's been in the works for 10 years, and most of the vendors have been involved for the last 5 years.
Has Neo4j been involved in the GraphBLAS design process and/or are you moving toward adopting the standard?
---//---
[1] Achieving 100M database inserts per second using Apache Accumulo and D4M [pdf] http://www.ieee-hpec.org/2014/CD/index_htm_files/FinalPapers...
Previous Discussion: https://news.ycombinator.com/item?id=13465141
[2] GraphBLAS Standard http://graphblas.org
[3] Jeremy Kepner http://www.mit.edu/~kepner/
[4] GraphBLAS: Building Blocks For High Performance Graph Analytics https://crd.lbl.gov/news-and-publications/news/2017/graphbla...
[5] Scott McMillan philjohn ↗ Yes, but your clustering is just read-only replicas. So you still have the problems that when your graph gets huge and you run out of space to scale each node up you're going to have a bad time. And it also ignores that when you're talking about scaling up to truly monstrous machines with TB of memory the cost is astronomical.
I agree, it's not an easy problem to solve, but saying nobody has solved it isn't accurate either - OpenLink (who I mentioned previously) solved it for their RDF engine. At a previous gig using that across an 8 node cluster we were seeing load rates of hundreds of thousands of edges (or in this case, quads) per second.
RDBMS has its strengths but it isn't suitable for every use case.
Btw, if you've ever stayed at a Marriott (or anyone they now own: Starwood, Hilton, Ritz Carlton) then you've used Neo4j in production (to book that room). If you've ever purchased a flight ticket, then you've used Neo4j in production (over 99% of all fare calculations are done with Neo4j). Etc.
https://www.youtube.com/watch?v=wdFYEuWWpzo
Pretty amazing stats in there!
You can use ODBC http://www.swi-prolog.org/pldoc/doc_for?object=section(%27pa...
You can use CSV http://www.swi-prolog.org/pldoc/man?section=csv
I've used the above methods.
For RDF, you can load with various methods even over http http://www.swi-prolog.org/pldoc/man?section=rdflib
I suppose one could have a prolog service that came up with suggestions, ordered by preference, and then attempt committing them in order to the actual transactional storage layer. But I don't see how you could avoid re-implementing transactions.
With an rdms you could look for options, and attempt a booking in a transaction, and have a fairly established way to resolve conflicts.
I've used Neo4j in the past and it seemed to be stable and efficient, not sure about how well it scales though.
Before, TitanDB (now JanusDB) in conjunction with the Tinkerpop stack was probably my favorite graph DB/stack, but not sure how seriously Datastacks (who owns TitanDB now) continues developing it. And JanusDB as a project didn't seem very active to me (I could be wrong of course).
That said, you can construct and handle graphs using relational databases, graph databases give you advantages in that they have (often) better indexing (i.e. O(1) lookups of vertices & adjacent edges) and come with graph querying languages (e.g. Gremlin), which make it much easier to work with graphs compared to SQL (you can use recursive CTEs to walk graphs on the database side as opposed the client side but complex queries are hard/impossible to write like that). I've written a graph DB abstraction layer in the past that also supports SQL backends: https://github.com/7scientists/vortex.
Fast forward 3 years and am back in the graph world, this time dealing with money laundering and fraud rings. It stumps me as to why graphs are not used more in the financial world. Financial transactions are no different from social interactions. So concepts such as community detection etc apply to the financial realm as well and in fact we are using such concepts to determine fraud rings. I may get back to neo4j.
20 of the 25 biggest banks in the world use Neo4j today, and fraud detection (specifically fraud rings) is a really popular use case. (So much so that if you search for "fraud detection" in an incog chrome window, https://neo4j.com/use-cases/fraud-detection/ is on the first page of google).
Btw, we've scaled out horizontally for reads since 2011 but our initial Paxos based implementation was shaky. Since then we've revised it twice and our third generation Raft based architecture is significantly better. If you add multi clustering support + causal consistency and a bunch of other improvements I think our horizontal scalability story is very different today from a few years ago. Lemme know if you try it out and disagree!
EPaxos[0] - flexible/minimal quorums + if you're willing to know ops are commutative you can fast-path, and you can flexibly quorum
WPaxos[1] - flexible/minimal quorums + dynamic partitioning via object stealing for spreading writes around
The main benefit of these is of course getting over raft's weakness of the leader being a SPOF.
If you're willing to go with causal consistency there's also CmRDTs (operation/command Replicated Data Types) which are generalizable to "pure operation based CRDTs"[2] which make the merging problem trivial and should in theory boil things down to replication lag while allowing writes at every node.
[0]: https://www.cs.cmu.edu/~dga/papers/epaxos-sosp2013.pdf
[1]: https://cse.buffalo.edu/tech-reports/2017-03.pdf
[2]: https://arxiv.org/pdf/1710.04469.pdf
One benefit I didn't think of before but has become clear: non-technical people find viewing and manipulating the graphs very easy, and explaining the links is much simpler.
Funny to me how tech is constantly a cycle of folks having no idea about the previous incarnations of their ideas.
It's an easier cultural change in thinking from rdbms to property graph, but also not a huge improvement in terms of what you can do vs an rdbms. Going with a full semantic graph where the relationship is also represented by a node with unlimited relationships, and moving and thinking in hierarchies and inference, is a complete cultural change with an impressive productivity and capability improvement not possible with rdbms. Allegrograph is a good example of a semantic graph which can handle trillions of triples.
I'd be happy to be proven wrong, I love graphs!
Before getting into Neo4j (and later joining the company), I'd worked with SQL databases a lot and, at one point, as a DBA. I spent the first few months trying to draw direct analogies between graphs and tables but after a while realised that I needed to "unlearn" a load of what I thought I already knew.
It's now pretty refreshing to be able to look from a higher vantage point at the different types of data store available, and work out which is the right one for each job.
It turns out that the graph is actually the best solution more often than you'd think!
http://www.swi-prolog.org/pldoc/man?section=semweb-scalabili...
https://franz.com/agraph/support/documentation/current/lisp-...
The other REALLY big advantage, the data model never changes! I literally let the graph, to a certain extent, write the code (that is, code is generated based on graph relationships on the fly)
And finally, you can have everything in the "same" location, you are always working with an "information" lake - no need to have many many different tables.
I don't see anything about a semantic graph database that would prevent it from being built on top of a graph database like Neo4j (and we are doing kind of that at my current company).
I've used it in production settings where it was NOT crushing a use case. (A couple small internal HR-related sites that would've been better suited to an RDBMS.) Based on that experience, my impression is that it would take a fairly specific graph-oriented problem to get me seriously thinking about paying the costs associated with something like Neo4J when compared to a traditional RDBMS. (Less mature compared to RDBMS systems, less well understood data model, less common query language, less support from other tooling.... all of which can be quite important to the overall costs of a system.)
It is still a research thing, but I am starting to see occasional papers on inducing relations in graphs using deep learning. If this proves useful, that should help the growth of graph databases and the use of knowledge graphs.
However, as another commenter ‘hardwarsofton’ said, just using Postgres is very often all I need.
I think we're in the early parts of an exciting journey connecting (no pun intended) graphs and AI. I'm personally really excited about connected feature extraction. I wrote a little bit about it here: https://neo4j.com/emil/80-million-series-e/ There's more in depth info here in this graphs & AI overview video from GraphConnect last month: https://neo4j.com/graphconnect-2018/session-topics/?topic=AI...
I'm also interested in peopels opinions of Neptune in that domain.
Mostly for medical reasoning on RDF OWL ontologies such as "Ontology Development and Debugging in Protégé using the OntoDebug Plugin"[1]
[1] https://www.youtube.com/watch?v=vHmC-rRuMYM
Granted, they won't let you do nearly as much as some advanced graph algorithms, but the ease with which you can use it in your operational data store is amazing. And with proper indexing, I could do a traversal in hundreds of milliseconds.
https://www.postgresql.org/docs/current/static/queries-with....
It's also however a bit of a dead end once you go beyond the basics. The costs of joins get worse the deeper you go, and "hundreds of milliseconds" is at least an order of magnitude slower than what Neo4j would do for you.
Once you take that major performance penalty, and then layer it into more complex graph algorithms or analytics, it gets really, really painful quickly. Granted, you might not notice this if you never needed to go further than 2-3 hops in a graph. But once you start working with graphs you're not going to want to stick to such basics.
More technical detail on the difference between a graph abstraction on top of another database, and a native graph database, can be found here:
https://neo4j.com/blog/note-native-graph-databases/
So say A points to B you have 3 tables right? Table 1 for A, Table(2) for B and a join table (3) showing that A points to B right? Why would you do that? What's stopping you from having one table that contains A and what A points to? So you only have 2 tables?
What if you have a node that can point to many items, a column can contain a list in postgres, so we can still have one Table containing your node data and a list of items they point to.
I'll concede that graph databases are easier to write query for, most people already struggle with basic SQL, let alone CTE and recursive CTE.
I'm yet to be convinced that a problem can't be reshaped and mapped on a traditional RDMS and yet remain performant.
So you can do that. Suppose you denormalize the graph into a single table. Either your denormalizing, and to connect A to many B's you have to duplicate A's data in the table, or you have the constraint that the A -> B link can only have a cardinality of 1.
This would be not a good choice for nodes with many relationships to other things, say for example "Person friended Person". It might work if the cardinality was somewhat capped, like say for example "Customer ordered Product" (a common denormalization).
For a real application, you're not going to have 1 of these. You're going to end up with 20+.
> I'm yet to be convinced that a problem can't be reshaped and mapped on a traditional RDMS and yet remain performant.
Any problem can be re-shaped to any database formalism. For that matter we can re-shape everything we're discussing for a straight K/V store like Redis. The expressive power of a database isn't at issue, because any database can store any dataset. Period.
More relevant questions are about performance and about conceptual fit for the problem. First on performance, have a look at the performance growth graph here:
https://neo4j.com/blog/oracle-rdbms-neo4j-fully-sync-data/
If you reason from the computer science of how these systems work, this result makes sense.
From the conceptual simplicity standpoint, that's kind of a matter of personal taste and application. Can you do it all with an RDBMS? Sure. But all of these different tech niches exist because sometimes you want more than one kind of tool for the wide variety of jobs you need to accomplish.
I'd argue that it's conceptually simpler to think of your graph as nodes and relationships, rather than to remember each time which node/rel set was denormalized into one table, which node label was split out into its own separate table, what that join table was, how the key naming differed between tables, etc. etc. etc. (Because once you have a non-trivial sized graph, you'll have a lot of these, and maybe you made different decisions at different spots).
The prize if you do remember all of that is that you get to write quite complex SQL to join together the datastructures for non-trivial traversals, because substantial graph use cases that you can answer on the basis of a single denormalized table are going to be rare.
In terms of conceptual simplicity, compare a recursive join SQL query to a cypher snippet like "MATCH (user:User {login:'bob'})-[:KNOWS*..5]->(foaf)". The equivalent SQL is...difficult.
It's just a "use the right tool for the job" situation at its core.
https://stackoverflow.com/questions/52674380/improving-postg...
Also MS SQL Server supports graphs natively: https://docs.microsoft.com/en-us/sql/relational-databases/gr...
Also, the official site says it's a commercial product. I wonder how many features are supported in community/opensource/free edition ?
The reason I use a graph is for consistency from my product level business logic to my implementation.
Basically, to solve my problem, I started with a set of english statements, which yielded a grammar, that I described as "objects and morphisms," (things and relationships) then implemented it in a graph - put a front end on it and built a product.
Graphs provide coherence to my problem. Could an RDBMS do this? Yes, but not without a complex intermediate query layer. I think of using a graph as analogous to specifying your problem in terms of a functional language instead of imperatively. The reason to do that is because your product is the result of maintaining consistency of an abstraction, like a DSL or a game, instead of just retrieving stored values, documents and their variations.
It's disruptive to a lot of orgs as well, since there is a lot of sunk cost in RDBMS experience, so I think the applications are all net new projects. I don't foresee anyone migrating to one, but I do see a point where majority of new products use one.
Once I learned Cypher and some common graph data modeling constructs, I found I could build more complex applications faster using Neo4j, largely because I found the graph model of my project's domain more intuitive and easier to work with than a relational database model.
Similarly, building and using GraphQL APIs has been a huge productivity win once I figured out how to build GraphQL services.
Of course, when used together, Neo4j and GraphQL have some great synergies, since it's graph all the way down ;-)
It's also worth pointing out that Neo4j has some great GraphQL integrations [1].
[1] https://grandstack.io/docs/neo4j-graphql.html
Lessons learned for me:
- start with very small datasets (mine was 10k nodes/40k edges/5 types and that still sent it to the moon in many cases)
- get to know the query planner right away
Cypher seems to reward a more deliberate and iterative approach to learning it rather than the ol' grip it and rip it. Once I settled down a bit it came together quite nicely.
Thank you for your insights.
We absolutely wouldn't be where we are without our community. From drivers and tooling to the huge amounts of support and goodwill that we see, it's a wonderful ecosystem to be a part of.
What we are seeing here is the 'commodification' of graph, a trend that happens in technology in general. Companies that launched ten years ago, on a massive investment with their own proprietary graph technology - I'm talking the likes of Twitter, Facebook and so on - today the same features could be implemented with a fraction of the investment. They'll do this by leveraging Neo4j.
This funding we'll broaden the reach of graph technology, while reducing the overall cost for individual organisations to adopt. Social networks, recommendation engines, fraud detection systems are all now easily within reach. Check out our own free and open-source recommendation engine, which was built on top of Neo4j, for example: https://github.com/graphaware/neo4j-reco.
We live in exciting times. While the commodification of what we call 'graph 1.0' is in progress, what Tesla's head of A.I. Andrej Karpathy brands "Software 2.0", that is the intersection of machine learning and software development is rapidly picking up pace. We're only at the beginning of the hype cycle on this. And guess what? IT is an established fact, is that graph is playing a central role in this transformation process.
We are proud to say that our organisation is at the forefront of using graph technology to derive insight and meaning from unstructured data - we call this GraphAware Hume. We're really excited about this!
As you can see we're are pretty passionate about graph technology, and Neo4j in particular, and in our opinion we're at the beginning of what is going to be a very transformative adoption. If you're thinking about exploring how graph might fit into _your_ organisation, of course feel free to reach out.
Disclaimer: GraphAware (https://graphaware.com/) is Neo4j's solution partner
Java will always be king of enterprise despite all the drama with the future of the JRE.
One would think that Neo4j probably is the most stable one? But it's unclear to me if the full version is open source or not? [1][2]
Has anyone tried several?
Has anyone tried Dgraph? [3]
I quite like it, but I haven't gotten to the 'running in production' part so I haven't experienced what it's like to actually manage or scale it, only its query language and setup.
[0] https://github.com/insanitybit/grapl
As a graph database, it has some non-typical tradeoffs. You can't easily discern incoming edges and there's no true node deletion. There's a pretty narrow happy-path where the DB works as advertised/expected, but it's just a fairly young DB from an understaffed startup. Probably worth waiting a year or two for the kinks to be ironed out.
[0] https://jepsen.io/analyses/dgraph-1-0-2
Congrats to Neo4j on the raise! I hope it changes the perception of US VCs w.r.t. graph DBs, who are falling behind the dev enthusiasm and readiness for adoption in this field.
As Jepsen report mentioned, it had identified 23 issues, 19 of which were resolved before the report released and another one right after. Dgraph has gone a long way since v1.0 release in terms of production stability. I'd recommend trying out the latest v1.0.9 release or the upcoming v1.0.10.
Dgraph itself is close to being launched in production at a few very big and well-known companies (that we can't mention publicly yet), who moved away from Neo4j to Dgraph. Needless to say, Dgraph's performance and scalability far exceed any other graph DB in the market.
Dgraph is tackling a lot harder problem of doing distributed joins and traversals, while providing distributed ACID transactions, synchronous replication and linearizable reads. The equivalent of Spanner, which can also do efficient joins (something relational DBs suck at, so technically more complex). There's no graph product out there like this or even a single paper which Dgraph is based on, rather we had to do original research to perfect this technology -- which is why it took time to build and stabilize Dgraph.
Badger, the underlying kv DB, itself was never found to have an issue. It is serving several petabytes of data in production use at various companies. We built Jepsen style bank tests for Badger, which run successfully nightly, and there's an open bounty of $1337 for finding any data loss bugs in Badger.
Dgraph is decently staffed (7 engineers) for a seed-stage startup, but we're definitely hiring and planning to grow in SF. No need to wait, this is the right time to run Dgraph in production.
And as far as I can remember from a former project the scalability is pretty limited (but this could have changed).
And there is a huge scalable graph stack with Cassandra - Datastax Enterprise Graph, Titan,JanusGraph (where Google is involved), Tinkerpop,etc.
The production readiness of neo4j is something I'm still not quite sure about.
It truly shines as an embedded graph db though. I wonder if there is a Blockchain story around neo4j (as a replacement for leveldb) that makes this more interesting. After all there is a lot of excitement around DAG based blockchain alternatives.
https://tinkerpop.apache.org/gremlin.html
Our schema involved taking physical assets/personnel and representing them as different labels: machine, factory, production line, user, usergroup, etc. We then drew complex relationships between different user/groups in the organization and the assets they were responsible for.
At first, we used a relational database, but it soon became difficult to go more granular than simply: user belongs to usergroup, usergroup belongs to client, client has factories, factories have lines, lines have machines.
As many have pointed out here, it's not that you can't do this with non-graph databases, it just requires a more complex query layer. Neo4j allowed us to represent complex business relationships as natural language, and that really helped us as the business scaled.
This may help non-graph folks understand the community a bit. Neo4j has a bunch of cool bits, and it's been a pleasure watching them bring two specific "aha!" moments to customers. Our tech helps teams build scalable visual workflows that include visual graph, so we're often brought in near the beginning of a graph project, and have repeatedly seen two situations where a DB at Neo4j's quality shines:
1. Performance: A teams starts using their existing data stores -- SQL, Splunk, etc. They'll get quite far. Often, however, they will hit some query that just cannot perform. E.g., for two bank accounts, all paths between them. For different DBs and workloads, these can be different things.
2. Ease: Asking for something like a 360 view around a device, user, patient, account, etc. is hard in sQL - you don't know what column, table, etc. to look at. Or imagine the above shortest-paths query. Cypher makes writing this stuff EASY, so in a world where a lot of people can barely do SQL, that's a superpower.
Neo4j has been broadening by entering the scaleout world, app dev world, and adding multi-modal & ML capabilities, which are all important things and help grow the eco-system. Congrats again!
I used to use OrientDB but moved away due to stability issues (long ago now so hopefully they have that sorted). I also just noticed they’ve been acquired by SAP!
For me graph layouts are conceptually superior vs relational when explaining to non techie users.
Practically however I now stick to Postgres - it’s ‘good enough’ (for what I’m doing) and has a heap of benefits in and of itself.
I looked st Agensgraph but I couldn’t get enough info on it, plus it is a custom version of Postgres (not a plug-in) and i think they recently switched to AGPL which makes it overall less exciting to investigate.
I know there are places where having a real graph db helps but I’ve not personally hit those scenarios yet.