HN Explorer

Ask HN: Are graph databases used in the wild?

8 points by jaxk ↗ HN
I am a front end guy reading up on databases for some backend task. It seems like graph databases try to manufacture a problem rather than address existing ones. They also always use either fraud detection or some such usecase that's niche among niche. That makes me wonder if businesses actually use/need them or its just a glorified academic project, but are there so many products in the space competing!! What's going on?

14 comments

[ 4.8 ms ] story [ 92.4 ms ] thread
malux85
Graph databases are used in many many places. Examining bank transactions, finding the shortest path through state machines, molecular reaction modelling, transactional arbitrage/trading, circuit modelling, fact modelling in AI, threat modelling in cyber security, network routing and large scale analysis, virus and sickness modelling, there’s so many more

These systems usually have specialised on-disk data structures and algorithms (e.g. path finding) that are multiple orders of magnitude faster than the equivalent data stored “Flat” in a column database, this difference in time and space efficiency isn’t “some glorified academic project” - but the concrete difference between practical and impractical

jaxk
yeah, I see these examples a lot so safe to say its niche and used by large banks, feds etc and not everyday businesses?
itguy1308
What do you mean with "everyday businesses"? I would not say that graph databases are niche...
thorin
I wouldn't say they are niche, but many people I work with would model these systems in relational or no sql databases rather than pick a graph database for such a specific use case. With the popularity of microservice approach, this may be changing. Traditionally applications were based around 1 or few single datastores.
jaxk
exactly, seems graph db just help reduce verbosity of sql query if cypher or something else is used.
malux85
No, because although individually each use case above could be described as niche (maybe) - there’s so many different ones, in so many different industries, there’s clearly a general use case.

I’ve been a developer with 18 years commercial experience now and I’ve seen graph databases in all size businesses as I’ve travelled the world, from 1-2 person startups to huge online casinos, heck, I’m a single person startup and I have two myself, one with 200,000,000 nodes and there’s no way a traditional database could do what that system is doing (I tried with Postgres, HBase on Hadoop, and Cassandra)

Sure they are not as common as traditional DBs, but they are far from niche, that’s why (as you said in your original post) there’s so many products in this space competing - they are actually a thing, they are actually useful, and for some projects, critical.

jaxk
yeah but as a novice dev I have never come across a job listing that needed a graph db expertise of any kind, there are tons that need nosql though, along with traditional rdbms of course, why is that?
[deleted]
(comment deleted)
chalcolithic
Actually upwork seeems to have plenty, and unless I'm mistaken they are all of "(re)write cypher query" kind.
chalcolithic
What is your graph db of choice? What are the more popular ones in your opinion?
randomopining
Graph DBs I believe are just useful when your data and the value you can get from your data is heavily reliant on relationships. This lets you construct a graph, and the field of CS/Math that has evolved around graph structures is implemented in a DB software that abstracts useful features in common functions etc.

Graph implies lots of nodes, which means lots of data. Only certain businesses use lots of data. Most small/medium businesses won't use it. That's why it seems like graphs are mostly in use at big tech etc.

unknown_error
Whether or not you use a graph DB under the hood, having access to a GraphQL API (https://graphql.org/) is much more elegant than having to join together a bunch of separate REST calls or SQL queries. It automatically traverses relationships for you based on the fields you need, and returns only the fields you ask for. So you get exactly what you need, no more and no less. Most GraphQL servers have a GUI explorer query builder too so you just click on the fields you need and it'll construct the query for you.

GraphQL can also run on traditional SQL databases, but I think at some point they hit complexity limits and performance issues because JOINs are hard for them, especially if the columns aren't indexed. In a proper graph database, relationships are first-class parts of any data model and there are no "tables" to speak of, just nodes and the arbitrary, complex relationships between them. It makes data modeling both more intuitive (if your data is naturally a graph) and more performant. Here's one take on it (https://developers.mews.com/intro-to-graph-databases/)

At the end of the day there is no magic cure-all for data storage. It depends on the data you're storing and the way you need to read and write from it. And business wise, it may not be worth it to rewrite 10 years of SQL databases to improve performance by a few percent. But if I were starting a new project from scratch, one that involves layers of interconnected data (say, a bookstore with connections between books, images, authors, customers, reviewers, reviews, inventories, third-party merchants, Goodreads entries, different versions for audiobooks and ereaders, multiple books in a series, etc.) it's the kind of thing that would lend itself well to a graph database in modeling, as long as the stack can also be performant and scalable enough for end users.

As a web dev "in the wild", we've offloaded the scaling problem to a vendor by choosing to use a headless CMS (GraphCMS, DatoCMS, Contentful, Prismic, etc.; there are many). Some of those use graph databases while others don't, but at the end of the day we don't really care as long we don't hit their complexity limits. They play DB admin, we get GraphQL or traditional REST APIs, and we can build a frontend on top of that.

mattcdrake
Graph databases are used by the IRS for (at minimum) detecting "patterns of abusive tax transactions"[0] and tax fraud[1].

The IRS also recently contracted Brillient to "define and prototype a graph database for the individual taxpayer"[2]. This is supposed to "enable IRS researchers to visualize complex relationships to improve compliance and enforcement."

[0] https://www.irs.gov/pub/irs-soi/09rescongraphquery.pdf

[1] https://www.aaai.org/Papers/Workshops/2005/WS-05-07/WS05-07-...

[2] https://www.brillient.net/news/brillient-awarded-new-task-or...