HN Explorer

24 comments

[ 1.5 ms ] story [ 41.1 ms ] thread
jauntywundrkind
Nice having this backstory (fantastic production value too, impressive start to this podcast). Dis-aggregating the responsibilities of the DB into multiple pieces just feels so logical, helps make sure each piece can scale. Deterministic Simulation Testing gets mentioned in the video & was way ahead of it's time here. https://apple.github.io/foundationdb/testing.html

Hacker News is here too! From July 2012 (78 points, 72 comments): https://news.ycombinator.com/item?id=4294719

For a general introduction, I enjoyed the recent submission How FoundationDB works and why it works: https://news.ycombinator.com/item?id=37552085 https://uvdn7.github.io/notes-on-the-foundationdb-paper/

chamomeal
I watched a super interesting talk about a testing library that was inspired but the foundationDB testing story.

It uses test.check (clojure version of quickcheck) to help you create valid mocks of stateful services to test against. I just love the idea and I hope to implement a small version of it in TS at work soon

https://youtu.be/dvHASrrQSzg?si=jF_sTFWL-wCwFNHA

For indexing purposes, the name of the talk is “Breaking the bank with test contract”, given by Allen Rohner.

vlovich123
What a great story and really interesting courage to double-down on improving the testing even when a critical flaw that testing should have found was found. Wish that they had managed long enough for Snowflake to keep them alive, but then we wouldn't have Antithesis as a service so silver lining.
majestik
I can't put my finger on it but there's a weird tension between the two Dave's in this video. Almost like Rosenthal is trying to impress or earn the praise of Scherer.

Is there a backstory between these guys / FDB?

philosopher1234
Does anyone know of cool things built with fdb? I’ve been aware of it for a while and it seems very cool but I haven’t seen a lot of details about how folks are using it.
jwr
I am moving my SaaS from RethinkDB to FoundationDB. It's a long-term project that needs to be done very carefully (thousands of people using the app), but the rewards are significant. Thanks to FoundationDB versionstamps, I'll be able to replace changefeeds with polling, simplifying the system, and also make things much faster along the way.

The consistency guarantees are phenomenal and writing software is much easier when you have strict serializability. Most people do not appreciate this because they do not understand the anomalies that you can get without strict serializable consistency.

mannyv
From what I understand one of the big IP ad tracking services (El Toro) is built on FoundationDB.
Quarrel
I'm excited by the idea of pgfdb*. I think it could be a great. postgres, with foundation replacing the transaction system to provide all the benefits that brings, horizontal scalability, automatic sharding, replication, and fault tolerance, and performance. foundationdb is 20+ years ahead of postgres in terms of distributed transaction theory.

* https://github.com/fabianlindfors/pgfdb

redman25
Arroyo uses it for their streaming database. Think streaming stateful aggregations. It’s pretty cool tech. Looks like they were recently acquired by cloudflare.
olavgg
Cognite built their dataplatform with foundationdb.
romanhn
Posted about this in the past, but what really got FoundationDB on my radar was a demo at a developer conference, back in 2014-ish. They had the database running across a bunch of machines, with a visual showing their health and data distribution. One team member would then be turning machines on and off (or maybe unplugging them from the network) and you could see FDB effortlessly rebalancing the data across the available nodes. It was a very striking, impressive presentation (especially as we were dealing with the challenges of distributed Cassandra at the time).

The beginning of this video has some of that: https://youtu.be/Nrb3LN7X1Pg

Nican
FoundationDB has been growing as my favorite database lately. Even though it is only key-value store.

Out of curiosity: what are the scale limits of FoundationDB? What kind of issues would it start to have? For example, being able to store all of Discord messages on it?

I see blog posts of Discord moving to Scylla and ElasticSearch, but I wonder if there would be any difficulties here.

heipei
ScyllaDB discontinued it's free and open source version, so I personally wouldn't build anything new on it.
eatonphil
There are a lot of strict limits so AFAIK everyone uses FoundationDB for fast, consistent, highly-available metadata while doing replication/storage of actual data elsewhere (such as in S3).

https://apple.github.io/foundationdb/known-limitations.html

That is to say it's more like part of your solution and not the entire stack on its own.

msy
Does anyone know how widely FoundationDB is now being used at Apple? I know they run a huge Cassandra cluster, does this serve a different use case?
ntqz
My understanding is that CloudKit runs on it.
ethan_smith
Apple uses FoundationDB extensively for iCloud services including CloudKit, with public documentation confirming it handles billions of operations per second across their infrastructure.
[deleted]
(comment deleted)
piokoch
I've looked on FoundationDB and on paper it looks great. But it never got momentum, like, say, MongoDB. Is this just a matter of hype or it is not that great as advertised?
gregoriol
So if it has been acquired by Apple, it's a failure, isn't it? Most things acquired by Apple get unmaintained or change completely, or disappear. Being "open-source" here doesn't bring any guarantees to any third-party user about maintenance or long-term life. It should be a serious no-go indicator for anyone willing to build something with it.