I believe Ceph [1][2] could be a good alternative. It can be self hosted and I believe some cloud providers also offer it. Here are some differences between S3 and Ceph [3].
Digitalocean Spaces uses Ceph as their backend and it is NOT stable (at least with s3fs), had to migrate away from it and could not do a full-stack on DO (aka "no AWS") and had to swap in Wasabi.
Protocol alternative? What are you specifically looking for?
It's object storage, the protocol is not that nuanced. Any other storage product will have it's own quirks. google cloud storage has an XML API that is similar enough and has signed urls, and just about every enterprise storage company has an "object" product.
You say protocol alternative, but assuming you're more concerned with AWS as the host than S3 as the protocol you might try https://github.com/minio/minio
If you do feel an aversion to the protocol then the rclone backend list would be a good starting point
The wording is a bit unclear, but I believe they were asking for an alternative to AWS that still uses the S3 protocol, not for an alternative to the S3 protocol.
Did you happen to submit this bug to Backblaze for triage? If not, I'll get it to the right people!
> When you're using an SDK, you can set the value of the x-amz-sdk-checksum-algorithm parameter to the algorithm that you want Amazon S3 to use when calculating the checksum. Amazon S3 automatically calculates the checksum value.
> When you're using the REST API, you don't use the x-amz-sdk-checksum-algorithm parameter. Instead, you use one of the algorithm-specific headers (for example, x-amz-checksum-crc32).
I hadn't considered it! Your username is appropriate, too much to do :D
My requirements are pretty low so I was likely to just let it slide...
...but with your help (and perhaps from others on understanding the pieces), I'll consider it!
I'm remarkably unfamiliar with S3 style of services, only occasionally trying to dabble
edit: The choosing of appropriate headers may come down to Mattermost/clients -- BackBlaze might be technically doing the right thing saying "I don't know what to do with these"
Please do. Backblaze lists x-amz-sdk-checksum-algorithm as unsupported [1]. Would be great to have it supported to be able to use it with Mattermost and other tools that use min.io for S3.
Backblaze is not a good choice for most things unless you were using it strictly for backup. Their S3 compatibility, as is most S3 compatibility, is a cruel joke last I looked at it.
I publish a mildly(?) popular offloading plugin for WordPress and adding Backblaze support was my biggest regret.
I'm not sure what "protocol alternative" means or what your goals are, but it's worth suggesting that pretty much every other big cloud provider offers an S3-compatible blob storage product. Are you looking for something you can deploy yourself or just an alternative to S3 itself in general?
If you are looking for an s3 protocol alternative it's probably swift. [0]
There is no widespread software adoption like for s3.
If you want something that is accessible via http and that's it take a look at seaweedfs? You can put files in over different gateways but access it over http. The rights management is probably not what you want. (You mentioned acls)
If you want some storage that you can access over http just setup anything and use a webserver like nginx in front of it...
The other commenters are right and you need to be more specific about your requirements.
34 comments
[ 3.0 ms ] story [ 84.8 ms ] thread[1] - https://ceph.io/en/
[2] - https://docs.ceph.com/en/latest/radosgw/s3/
[3] - https://www.lightbitslabs.com/blog/ceph-storage/
https://docs.ceph.com/en/latest/radosgw/swift/
It's object storage, the protocol is not that nuanced. Any other storage product will have it's own quirks. google cloud storage has an XML API that is similar enough and has signed urls, and just about every enterprise storage company has an "object" product.
I've only used it as a fairly straight forward object store though, so not sure about privileges/permissions (etc).
If you do feel an aversion to the protocol then the rclone backend list would be a good starting point
https://rclone.org/overview/
I like recent (v3) SMB over the network personally.
For public access, something like webDAV is probably the lowest hanging fruit.
However its still HTTP based, so not great for speed or partial writes.
HDFS is not worth the hassle, same goes for ceph, its a lot of work for slower worse performance than ZFS/pNFS.
If you want speed and posix, then something like GPFS from IBM is where to go, failing that, lustre, which you can get from AWS if you want to try it.
Tangentially this is lamenting at the lack of choice
Last night I tried to set up BackBlaze (S3 'compatible') behind Mattermost.
This doesn't go very far when clients use/rely on headers that specifically mention vendorization
BackBlaze won't work with clients that try to send x-amz-sdk-checksum-algorithm, Mattermost taught me this
TLDR: S3 isn't always S3
> When you're using an SDK, you can set the value of the x-amz-sdk-checksum-algorithm parameter to the algorithm that you want Amazon S3 to use when calculating the checksum. Amazon S3 automatically calculates the checksum value.
> When you're using the REST API, you don't use the x-amz-sdk-checksum-algorithm parameter. Instead, you use one of the algorithm-specific headers (for example, x-amz-checksum-crc32).
https://docs.aws.amazon.com/AmazonS3/latest/userguide/checki...
My requirements are pretty low so I was likely to just let it slide...
...but with your help (and perhaps from others on understanding the pieces), I'll consider it!
I'm remarkably unfamiliar with S3 style of services, only occasionally trying to dabble
edit: The choosing of appropriate headers may come down to Mattermost/clients -- BackBlaze might be technically doing the right thing saying "I don't know what to do with these"
Please do. Backblaze lists x-amz-sdk-checksum-algorithm as unsupported [1]. Would be great to have it supported to be able to use it with Mattermost and other tools that use min.io for S3.
[1] https://www.backblaze.com/b2/docs/s3_compatible_api.html
I publish a mildly(?) popular offloading plugin for WordPress and adding Backblaze support was my biggest regret.
I was truly surprised to see Amazon branded headers in something claiming to be 'standards compliant'
They may be optional, but their presence simply complicates things; particularly when clients opt into them.
It devalues what is common/reused! Either locking you into Amazon or another protocol entirely
I switched to Wasabi and am using the C++ AWS SDK. I’m impressed by their performance
Usage is domestic fibre gigabit symmetric from a 2013 MacBook Pro.
There is no widespread software adoption like for s3.
If you want something that is accessible via http and that's it take a look at seaweedfs? You can put files in over different gateways but access it over http. The rights management is probably not what you want. (You mentioned acls)
If you want some storage that you can access over http just setup anything and use a webserver like nginx in front of it...
The other commenters are right and you need to be more specific about your requirements.
Edit added link and some text I somehow missed [0] https://docs.openstack.org/api-ref/object-store/index.html
[1] https://docs.openstack.org/swift/latest/
1. "alternative implementation with the AWS S3 protocol"
2. "alternative protocol with similar features to AWS S3"
?
Edit: Disclaimer I work at Cloudflare