Most of the pain here is the typical set of issues people run into trying to make S3 a filesystem as-is, common with S3FS-family approaches.
ZeroFS (https://github.com/Barre/zerofs) is 9P/NFS/NBD over S3 on an LSM. Point stock go-git, or just /usr/bin/git, at a mount and skip the gymnastics. Rename is a metadata op in the keyspace, so you get it atomic on any S3, no Tigris-specific X-Tigris-Rename needed.
Different point on the spectrum, but less square-peg, also most probably much, much faster (it works great on linux-sized repos) :)
This was really thought provoking — it made me realize that Git just happens to use a filesystem for persistence, but doesn’t necessarily have to. A POSIX filesystem might not even be the best way to store a git repo. Makes me wonder: what else could speak Git + POSIX? Redis? Postgres? IPFS is a fun one — it’s already content addressed.
I've seen a few people on HackerNews swear by Fossil ( https://en.wikipedia.org/wiki/Fossil_(software) ) which uses SQLite as the backend instead of the filesystem. There are a few other approaches as well
I did something similar, though a full reimplementation of a git and git-lfs library in Elixir. Still a work in progress though as the S3 backend isn't quite complete and there are performance problems doing some git things through S3.
9 comments
[ 2.9 ms ] story [ 25.2 ms ] threadGo Git and Dulwich and friends are indeed fun tech.
ZeroFS (https://github.com/Barre/zerofs) is 9P/NFS/NBD over S3 on an LSM. Point stock go-git, or just /usr/bin/git, at a mount and skip the gymnastics. Rename is a metadata op in the keyspace, so you get it atomic on any S3, no Tigris-specific X-Tigris-Rename needed.
Different point on the spectrum, but less square-peg, also most probably much, much faster (it works great on linux-sized repos) :)
[1] https://git-annex.branchable.com
https://anvil.fangorn.io/fangorn/ex_git_objectstore
The documentation isn't quite correct, but it's getting there
Not exactly the same thing, but along the same lines of "git and s3 are both object stores, why not use s3 as git storage?"