While not obvious from the article, it appears that they want something S3 like, but isn’t from Amazon, and possibly want to self host it. The article could be much more clear about the goals
Ah, thanks. Yeah I was confused because in his long list of vendors he didn't mention Wasabi, Backblaze etc. It appears that I do not know the context of his post.
I think we get a "S3 clone" about once every week or two on the Golang reddit.
It strikes me as a classic case of "we need all the interested people to pull in one project, not each start their own". AI may have made this worse then ever.
I only recently realized how much I like using Cloudflare more than AWS :) R2 (their version of S3) is no exception. Much more pleasant figuring out how to use and configure it in Cloudflare than the craziness inside AWS.
This is an interesting write up, but I'm curious about the use case. If you don't need to scale, and you don't need to replicate anything, why do you want S3 specifically? Are you using a tool that wants to write to something S3-like? Do you just like reading and writing objects via HTTP POST and GET? Are you moving an app to or from the cloud?
Since this has come up 4-5 times on the thread already, the clear subtext of this post is that this developer wants to build to the S3 API, but run their storage locally --- maybe for testing reasons, maybe for data hygiene reasons, maybe for performance reasons. So things like "what about Hugging Face's object storage product" don't really answer their question.
Yeah, but that missing context is super important.
If they want it for local dev work, that's pretty different from wanting a high-performance air gapped object store without rewriting clients.
They seem to know what they're doing (having complained about a methodology problem in MinIO), and yet don't personally want to throw their hat in the ring not maybe pay anyone...
It feels like there is more missing context than that:
> I just want S3. My needs are pretty basic. I don't need to scale out. I don't need replication. I just need something that can do S3 and is reliable and not slow.
What does reliable mean, without replication, and no mention of backups?
It doesn't need to care about POSIX mess but there is whole swathes of features many implementations miss or are incomplete, both on frontend side (serving files with right headers, or with right authentication) and backend (user/policy management, legal hold, versioning etc.)
It gets even more messy when migrating, for example migrating your backups to garagefs will lose you versioning, which means that if your S3 secret used to write backups gets compromised, your backups are gone vs on implementation that supports versioning you can just rollback.
Similarly with password, some will give you secret and login but won't allow setting your own so you'd have to re-key every device using it, some will allow import, but only in certain format so you can restore from backup, bot not migrate from other software.
The author is also not really clear on what they are actually needing.
If they just want webfile interface then a webserver with simple auth and webDAV would work more than well enough.
The problem is that they then go onto talk about lots of projects that all have posix interfaces. Which is slap bang into shared filesystem land.
S3 is not a filesystem, and nothing shows that more than when you use it as an object store _for_ a filesystem.
Depending on the access requirements, if you're doing local to local, then NFSv4 is probably more than enough. Unless you care about file locking (unlucky, you're in shit now)
I don't think that is what they are looking for. They just want something with an s3 compatible API they can run on their local network or maybe even on the same host.
I wanted to try NVIDIA’s aistore for our datasets, but I couldn’t figure out how to get a small version up and running so I gave up (a few years ago, today I’d get an LLM to show me how k8s works).
Settled with SeaweedFS for replacing minio and getting a good chunk of S3 feature parity. I wonder about the problems OP is posting about. Never seen that behaviour but usually only having a bunch of smaller files.
Call me crazy, but wouldn't 15 minutes on GLM 5.1 produce a working implementation? I haven't looked at the code, but a non-production-grade Go implementation can't be that complicated.
Edit: Minio is written in Go, and is AGPL3... fork it (publicly), strip out the parts you don't want, run it locally.
I did exactly that. 15 mins for initial implementation was about right. And it seemed fine at first glance.
Then I decided to run the Ceph S3 test suite against it. So many issues. Think it passed 3 tests on first run out of I think about a hundred. Took another couple of hours to get it to a state that is even vaguely passable for non production use.
Got something vaguely workable but even after many hours I can’t say I super trust it
> implementation can't be that complicated.
S3 has a fair bit of bloat in spec that nobody seems to use and it’s not clearly delineated which is core and what’s ahem optional. I ended up relying on the LLM to figure out what’s core and that ended up missing stuff too so that needed a couple of feature iterations too
67 comments
[ 0.23 ms ] story [ 91.4 ms ] threadhttps://canonical-microceph.readthedocs-hosted.com/stable/tu...
Its also not that useful even if you have enough machines to run it properly.
NVME and zfs is fast enough for virtually anything now. With snapshot and snapshot sending you get decent backups for half the hardware cost of ceph.
It strikes me as a classic case of "we need all the interested people to pull in one project, not each start their own". AI may have made this worse then ever.
Disclaimer: I work at HF
If they want it for local dev work, that's pretty different from wanting a high-performance air gapped object store without rewriting clients.
They seem to know what they're doing (having complained about a methodology problem in MinIO), and yet don't personally want to throw their hat in the ring not maybe pay anyone...
Context matters!
> I just want S3. My needs are pretty basic. I don't need to scale out. I don't need replication. I just need something that can do S3 and is reliable and not slow.
What does reliable mean, without replication, and no mention of backups?
It doesn't need to care about POSIX mess but there is whole swathes of features many implementations miss or are incomplete, both on frontend side (serving files with right headers, or with right authentication) and backend (user/policy management, legal hold, versioning etc.)
It gets even more messy when migrating, for example migrating your backups to garagefs will lose you versioning, which means that if your S3 secret used to write backups gets compromised, your backups are gone vs on implementation that supports versioning you can just rollback.
Similarly with password, some will give you secret and login but won't allow setting your own so you'd have to re-key every device using it, some will allow import, but only in certain format so you can restore from backup, bot not migrate from other software.
QS3 (Quite Simple Storage Service) for the barebones. Bucket/Object CRUD. Maybe: Multipart Uploads. Presigned URLs.
S3 for Object Tagging, Access Control Lists, etc.
S3E (enterprise? extended? elaborate?) for Object Lock & Retention (WORM compliance, Legal Holds), Event Notifications and so on.
* S4: Stupid/Silly Simple Storage Service
* S3: Simple Storage Service
* S2: Storage Service
If they just want webfile interface then a webserver with simple auth and webDAV would work more than well enough.
The problem is that they then go onto talk about lots of projects that all have posix interfaces. Which is slap bang into shared filesystem land.
S3 is not a filesystem, and nothing shows that more than when you use it as an object store _for_ a filesystem.
Depending on the access requirements, if you're doing local to local, then NFSv4 is probably more than enough. Unless you care about file locking (unlucky, you're in shit now)
You failed to answer why you even need s3... Why not a filesystem? Full stop. The entire point of s3 is distributed.
I'm a simple man, I just need edge delivered cdn content that never fails and responds within 20ms.
https://github.com/floci-io/floci
"firt commit" [0] was less than a month ago and added 51k lines in a single commit.
0: https://github.com/floci-io/floci/commit/61433f59ab995e9eaeb...
Edit: Minio is written in Go, and is AGPL3... fork it (publicly), strip out the parts you don't want, run it locally.
Then I decided to run the Ceph S3 test suite against it. So many issues. Think it passed 3 tests on first run out of I think about a hundred. Took another couple of hours to get it to a state that is even vaguely passable for non production use.
Got something vaguely workable but even after many hours I can’t say I super trust it
> implementation can't be that complicated.
S3 has a fair bit of bloat in spec that nobody seems to use and it’s not clearly delineated which is core and what’s ahem optional. I ended up relying on the LLM to figure out what’s core and that ended up missing stuff too so that needed a couple of feature iterations too