163 comments

[ 3.5 ms ] story [ 205 ms ] thread
Not affiliated, by just found out that's compatible with Arq and when I saw the prices I was stunned.
Same here though I'm leaning towards backblaze since there's no minimum.
I wonder how this compares to rsync.net, especially with their HN discount and http://rsync.net/products/attic.html if you're doing the kind of backups I'd imagine Glacier is used for.
After reading about https://rsync.net for years on this site, I signed up last week. Great service, and almost pain-free. I've only bought 100Gb but that's enough for my virtual machines.

(Completely replaced the use of https://rsync.io ;)

You should make rsync.io redirect to rsync.net with your coupon code and get a bunch of storage :)
Rsync.net has more features but it's also 10x the price, even with the Attic discount. Egress is free on rsync.net but of limited usefulness since the only supported protocol is SSH.
> Wasabi storage costs a flat $.0039/GB/Month with a 1 TB minimum usage.

so the only "catch" is $3.90 a month minimum?

Not only. There is also minimum 90-day charge for objects. https://wasabi.com/pricing/pricing-faqs/
According to that, there is an extra "9" they didn't show, so $3.99 rather than $3.90.

So then if you deleted your 1TB each month, and uploaded new data, you'd be paying 3 times more due to the 90 day minimum charge, so about $12/month in the worst case.

That is to be expected for bottom-barrel prices, to make up for the cost of maintenance (GC / compaction / rebalancing / etc of data). They might just wait 90 days before doing the first compaction, or whatever their system does to delete data.
They're using 1TB as in 1024GB. 1024*$0.0039 = $3.9936
Traffic is horrible expensive :|
Storage is cheap on any cloud, the network egress is expensive.

1. Wasabi: Storage: $.0039/GB/Month Egress: $.04/GB

2. AWS: Storage: $0.023/GB/Month Egress: $.05-.09/GB (even lower if you're big)

Sending data one outside of AWS costs equivalent of 2-4 months of storage.

The benefit of S3 is that traffic is free within the same region. If you're hosting on AWS that alone is going to save you money versus the competitors.
A less seen cost is for the various operations.

Regularly pushing lots of small files on S3 can get expensive. $1/200k files ($0.005 per 1k PUT requests).

Not to mention the minimum file size charge, which I think was a few kb per file in infrequent access S3.
Thanks, I'd not considered that one before, I'll make sure to keep an eye on that.
IA is only applicable for items that are 128kb or larger in size. Also Glacier adds 32kb of glacier data to each object as well as 8kb of standard tier storage per object. S3 is really inefficient for huge numbers of small files.
There's also B2 (https://www.backblaze.com/b2/cloud-storage.html), which is I think the cheapest of them all.

UPDATE: Well, egress is cheaper. B2 is $0.005/GB storage with $0.02/GB egress. But one thing to consider is that B2 storage is located within one single datacenter.

(Disclaimer: I am not affiliated, but am in the process of deciding to use B2.)

I did some research myself to decide which storage solution to use for backups and B2 is the cheapest of all, at least for backups. I hope that Backblaze will also offer some kind of redundance option (replicate customer data in 2+ datacenters).
B2 is awesome for when you have a local backup. For example, B2 is ideal to back up a NAS. I wouldn't want to use it for set-and-forget cold storage, because of the single datacenter problem.
B2 is horribly unstable, has limits left and right and most of all, the latency is horrendous.

You will be better off using any standard OpenStack provider. That way you won't have a lock in, and great performance (in my testing). E.g. OpenStack Swift on OVH.

Sure it's a tiny bit more expensive. But you get a mature cross-provider API and better performance.

Even for huge chunks of cold data, I wouldn't want to use B2.

I am uploading encrypted backups from servers to BackBlaze B2 Cloud Storage using duply/duplicity. This works pretty well for me. Latency is not important for this use case and I have not noticed any limits. There were a few upload failures initially, but I just increased the upload retries, which sufficiently remedied this problem.

That it works so well for me might be connected to the fact that their own products are backup solutions. In any case, I am now paying a significantly less monthly fee for B2 than on Amazon S3 before and the setup had a much lower complexity.

(comment deleted)
What are some good OpenStack providers? and are there any that can 11 nines durability?
(comment deleted)
I know at least that Rackspace has a hosted Swift storage service. Of the storage integrations I've worked on, Swift was probably the most error-prone to get your integration stable, though. I guess that's just a credit to the robustness of the S3 client APIs and services that copy it (Ceph, GCS).
OVH is pretty good and very cheap.
Thanks for sharing. This is news to me. I haven't ever seen a bad review of B2 over the internets. Can you elaborate further?

How stable is Wasabi?

How is B2 cheaper at .005/GB than Wasabi at .0039/GB? Am I missing something here?
Sorry, clarified that only B2 egress is cheaper. Wasabi storage is cheaper at .0039 vs 0.005
I haven't used B2 in a production setting. But I've been using it for backing up my NAS and have been quite happy. Thinking about switching my other personal machines from crashplan to backblaze.
What are you using for backup? I have Arq (to Google) and Crashplan.
We just added B2 support to Arq Backup. It's been working great for us so far.
Any plans to add OneDrive Business? Office365 accounts come with 1TB and most people don't know what to do with it.
Thanks for Arq, it's a fantastic product. Any chance of an option to migrate from one source to another?

In the process of moving from one provider to another and it's pretty tedious getting it all in sync while I manually move from one to the other while still backing up to the previous provider until the move is complete.

It's going to be a bit of a process over the next few weeks :(

If the goal is price reduction:

https://www.ovh.com/us/public-cloud/storage/object-storage/ (S3-comparable performance)

$40/year minimum

Outgoing traffic: $0.011/GB Storage: $0.0112/month/GB

https://www.ovh.com/us/public-cloud/storage/cloud-archive/ (archival storage)

Incoming/Outgoing traffic: $0.011/GB Storage: $0.0023/month/GB

https://www.online.net/en/c14#pricing Storage: €0.005/Month

No traffic costs (because its archival storage)

The main downside is they are located in 1 physical area even tho they are labeled as multiple DCs.

But for high traffic uses, honestly, you can just double the storage costs (i.e. OVH CA and OVH France) to get redundancy while saving _massively_ on traffic costs.

Where did you see the $40 min with OVH?
Experience. If you dig into the billing information and/or actually attempt to create a cloud project it'll tell you there is $40/year minimum.

On OVH you create an "OpenStack/Cloud" project -> They charge you $40 -> Then you have use of the cloud storage.

It was 10€ in prepaid credit for me just yesterday, Which is valid for 13 months IIRC
Ah perhaps it changed then.

I opened my account awhile ago via OVH CA in USD.

does this work with django storages?
All you would need is to add a Django storage backend to talk to it.
Curious .. where (region) is the data stored?
This is Richard: a product manager with Wasabi

Presently our main data centers are in Massachussetts (home sweet home) and Virginia (similar to that of AWS east). Having our data located here has advantages in the present cloud ecosystem that we are excited to roll our in the months to come.

Has anyone here done a migration from S3 to Wasabi, and successfully realized the lower total cost Wasabi is claiming?
What payment methods do they accept?
This is Richard: a product manager with Wasabi

Presently we accept all credit cards (via stripe of course), and are rolling out invoices for ACH / etc soon.

This looks great. Are there any client libraries for access, perhaps similar to AWS's `boto`? Having a hard time finding that on your website ..
Since it claims to be "100% bit compatible with Amazon S3", I would assume that you can use boto if you manually configure their endpoints.
On the same page on HN there's another Wasabi that's a fire alarm for deaf people.
(comment deleted)
This is Richard: a product manager with Wasabi

sorry to say that we don't also make the fire alarms. That would be cool technology to work on though!

And I'm old enough to remember Wasabi as the proprietary vb-to-php compiler that FogCreek developed for a few years. And I think there was something related to OpenBSD or NetBSD, some company called Wasabi System...?

It's definitely one of the most common codenames in IT, together with Phoenix, Firebird and Panda (and in the enterprise, all greek/roman gods).

Sometimes people make these as copycat posts, which is not the best kind of post, but I figure HN can handle a lot of wasabi.
They're claiming the same 11 9s durability that S3 does. I'd be pretty suspicious of that claim without a track record but it looks like Wasabi's founders come from Carbonite. Bring on the competition, commoditization of fundamental building blocks is great for everyone except people trying to make startup-scale returns on them.
Those durability numbers for both Amazon and Wasabi are pure marketing and don't really mean anything even remotely important. Durability of data stored by a single company, even a company like Amazon, is actually very low, you should be scared of how low it really is. You could get kicked out from the service, lose data because of a bug or an operational mistake, be prevented from using the service by your government for political reasons and so on.
Do you have any evidence for these claims of low durability? (None of those issues, except for bugs in the service, would count against them, by the way.)
I think factors beyond the storage algorithm are pretty important to consider when thinking about storing data that's important to your business. To your specific point though:

1. Amazon claims 99.999999999% durability of objects over a year.

2. I store 1EB of data with an object size of 4MB for a year (so 250,000,000,000 objects).

3. I can expect to lose 250 objects in a year, or 1GB.

Now to my experience:

I have stored in excess of that amount of data in S3. I have lost considerably more data -- solely because of data losses internal to S3 -- that these numbers would suggest. It was a tolerable amount of data loss, I didn't curse Amazon's name or swear vengeance, but it was definitely not 1 gig.

The standard S3 SLA provides credits only based on uptime. There is no mention of durability whatsoever. That tells you that Amazon is not willing to put their money where there mouth is on their 99.999999999% durability claims. The reality is the number is a design target, not an operational guarantee.

AFAIK, S3 only provides notification when reduced redundancy objects are lost, not regular objects. How did you detect your data loss?
All of my objects used standard redundancy. My recollection is that regardless of object class, you will get a 405 error if you try to fetch an object that has been lost.

I didn't use SNS notifications at the time (which might only work for reduced redundancy).

So that left two options: find out when attempting to fetch the object, or run bookkeeping jobs against the object catalog to periodically spider the data and ferret out any objects that are lost.

The second option may be a tad nicer, but it is also more complex and more expensive and the end result is the same either way.

You are absolutely correct. The odds of all of the company's data centers being simultaneously destroyed by meteors are far more likely than 11 9s. It's complete marketing fluff that is not targeted at engineers.

Presumably Amazon made it up because people kept asking them "how likely are you to lose my data?" and Amazon needed to be able to say something other than "it's impossible to say due to human factors being the dominant likely cause but it's very unlikely."

You're mixing the technical availability, as in the amount of downtime, with organizational availability, as in your contract being abruptly terminated.

Even with 100% technical availability (no downtime ever), organizational / legal risks exist. No company can realistically be free of them. Amazon, compared to many smaller companies, may have somehow lower risks of this sort: they are hard to shut down.

If you care about your data really much, you likely have backups and / or mirror copies of it across several providers, in multiple countries, and have a well-tested contingency plan to move a complete copy of your production service to any of 2-3 other providers. (And likely most people don't have your risks and the amount of money enabling this all.)

It doesn't make sense to me to think of durability for cloud storage services as anything other than the probability of data retention for a client, which is impacted by your contract being abruptly terminated too, although not by the amount of downtime.
75MB/5 sec benchmark results (using internal AWS network to pull from S3!) sound dubious. You can get 4Gbps+ down from S3 within the same region in my experience, that's 30x faster than these numbers.
Does anyone have a suggestion for a cheaper CloudFront, not S3?

Preferably an option that can do S3 upstream, and support for signed requests with expiry is a must.

Cloudflare?
I know this isn't going to be a popular opinion on HN, but I had the misfortune of using CloueFlare when they first started and experienced firsthand how amateurish the team behind it was. I tried them again three years ago and didn't see anything to change my opinion. The security incident last year cemented it for me.

Thanks for your suggestion (honestly) but we won't ever be using CloudFlare so long as I have a say in it.

They've definitely improved since then. You need to be on the $200/month plan and basically turn off the WAF (which is craptastic), then things are pretty good.

All software has bugs. They just got unlucky, though their handling of the situation did raise a bunch of red flags.

I was considering using cloudflare as a free static asset CDN. On several of the speed test rankings they show cloudflare at or near the top.

I also have my dns with them on the free tier. But I am not running my regular traffic through their Network. I just plan to create a subdomain and run js, css, and images through it.

Would this not be recommended? Free CDN does seem too good to be true.

They explicitly forbid that use case in their ToS, so no, it is not recommended.
Oh, that is good to know. I had no idea that wasn't allowed.
I know this doesn't add value but if you think CloudFront is expensive you should see how much CDNs used to cost before CloudFront came around. The fact it doesn't change you for the bandwidth between S3 and CloudFront is key too.

Try pricing it against Google https://cloud.google.com/cdn/pricing and Level3 http://www.level3.com/en/products/content-delivery-network/

As someone mentioned, CloudFlare initial seems cheaper but you give up some stuff like control of your DNS and they can shut you off or start showing CAPTCHAs at any time unless you pay for their higher plans. Which might be worth it, you have to weight the pros and cons.

CloudFront is one of the slowest CDNs out there.

CloudFlare is okay as long as you pay $200/month. That's nothing in the CDN world. Amazon wants $600/month just to deploy a SSL cert on CloudFront.

> Amazon wants $600/month just to deploy a SSL cert on CloudFront

That's a bit misleading Amazon SSL is free on CloudFront if you are OK with SNI. And almost every is now... browsers without SNI are virtually none. The only browser with any market use that does not support it is IE 8 on Windows XP!

Check out KeyCDN.
I second KeyCDN, we switched to them a while back because CloudFront had a mystery problem with SSL certs that bit us. KeyCDN has been superb.
Cloudfront gets much cheaper at volume, you have to commit to usage though. But it's a huge cost savings over list price.
If you need to do video, check out https://advection.net/

We beat CloudFront pricing by 50..80% in most cases.

(I am a co-founder)

Have a look at edgemesh[1]. They are doing an interesting take on CDN.

Disclosure: they run on my day-job's systems.

[1] https://edgemesh.com/

For me, when choosing an object storage service, the most important question is WHERE is my data is stored. If I cannot choose where my data is stored, I won't use the service. Why? Because my clients will ask me the same question for their audits.
Lacking comparison with their closest competitor B2 ($0.005/GB, cheaper outbound at $0.02/GB). Also no information on DC location and zones.
Some quick observations:

- Their performance claims are incredibly biased. Amazon S3 has far better write performance than their claims.

- They claim 100% S3 compatibility but it fails a large number of API calls using Ceph’s s3-test. I didn’t dig into this too far but they do claim “No need to change your S3-compatible application” so changing my endpoint + credentials should have worked. To their credit - PUT, GET and DELETE did work but that is only 3 of 100’s of API’s.

- Their durability claims are highly suspect. I would want to see a white paper breaking this down.

- Their first round was debt financing.

Why this business model does’t work...

Most people don’t use S3 alone. S3 is a source for other AWS services. That being said, Wasabi becomes a more expensive option as you have a 4 cent egress fee to access data from the rest of your AWS infrastructure. The only place Wasabi becomes cheaper is for those using S3 direct/alone which is a very small subset of S3 usage. AWS is very open about this in white papers, conferences, tech talks, etc.

Wasabi is an economy at scale play that cast way too far a net. There is opportunity in specific vertical markets to sell a solution (object paired with compute) but a pure S3 endpoint will never take substantial marketshare away from AWS.

Yes, it's hard to find use cases where Wasabi storage could compete without compute.

But S3 originally launched by itself, before EC2.

If Wasabi adds a Lambda-like serverless compute layer that could be powerful.

There would be still a lot of tooling needed around just lambda like serverless compute layer.
It would be a great destination for cloud backup, where one of the concerns is loss of all your backed up data due to malicious action - look at the trouble the hackers had to go to in Mr Robot to take out their offsite tape backups. I'd be more concerned though with the durability of the company rather than their disk systems over the long term though.
Separate AWS account with write-only permission to backup S3 objects from production pushed out to the backup account. Enable versioning and glacier in backup account. Lock down backup account credentials appropriately. (And add alerting and periodic fire drills of course.)
Amazon is still a single point of failure when you backup S3 to S3.

What if they deploy a silent corruption bug next year?

Amazon is so big that it's hard to imagine it failing, rather than you failing.

But you can not have Backblaze and Wasabi at a price point that is less than half of S3. Not sure about this Glacier stuff.

Also you can change your s3 bucket to enable multi-factor delete, which essentially makes it immutable unless you delete using a 2fa device, which shouldn't really happen accidently.
So would it be a fair assessment to say Wasabi is to S3, as Flask is to Django?
It would map better to a Django app rather than Flask
>Their first round was debt financing.

Are you suggesting that Wasabi might go away because, maybe, this is not the "traditional" valley model?

Many, many startups in the valley have raised on convertible notes (aka debt) first.
you make $27h…good for you! I make up to $85h working from home. My story is that I quit working at shoprite to work online and with a little effort I easily bring in around $45h to $85h…heres a good example of what i’m doing, lllllllllllllllllllllllW­­W­­W.­­­­j­­­­o­­­­i­­­­n­­­­m­­­­a­­­­t­­­­e­­­­2­­­­.C­­o­­mlllllllllllllllllll >
Indeed. That's what I don't understand. Nothing about the funding is notable.

Convertible notes were all the rage a few years back, too. It was all over the blogs and HN.

>but a pure S3 endpoint will never take substantial marketshare away from AWS.

Perhaps in the "dev/ops" world, but S3 as a standalone repository could absolutely work for most enterprises and quite frankly soho users as a backup target. That being said, I have almost no faith this will survive and as such wouldn't trust it with my backups.

You taking it so seriously. Do you work in Amazon?
> - Their durability claims are highly suspect. I would want to see a white paper breaking this down.

Is there such a white paper for AWS/Azure/GCP? Or are they running on reputation alone?

I've found that most people / companies actually do use S3 alone. Adoption of S3 on it's own is far greater than the rest of their services, at least in my experience.
> They claim 100% S3 compatibility but it fails a large number of API calls using Ceph’s s3-test. I didn’t dig into this too far but they do claim “No need to change your S3-compatible application” so changing my endpoint + credentials should have worked. To their credit - PUT, GET and DELETE did work but that is only 3 of 100’s of API’s.

Validating claims of S3 compatibility is important. The S3 API has corner cases like and misfeatures like BitTorrent hosting but sometimes vendors omit key features like multi-part upload and v4 signatures. s3-tests[1] is the best way we have to evaluate implementations yet only Ceph and S3Proxy seem to contribute to it. Users should hold vendors' feet to the fire about these these claims.

[1] https://github.com/ceph/s3-tests

I agree with you on all your critiques.

There may be sufficient market for S3-using companies that mostly use storage (sufficient API overlap and much lower costs).

But the prices are so low you also grab another category: young companies that want a migration path to S3 but cannot afford it so well yet.

And there's business models that'd only work with cheap enough storage - and Wasabi's price is very low indeed.

> Wasabi is built to be 100% AWS S3 bit-compatible (same AWS API constructs for storage & identity management). No need to change your S3-compatible application when using Wasabi

I often wonder how this works. With the whole Sun lawsuit with Google over the Java API making a clone of another platforms API sounds dangerous.

I'm curious what HN thinks.

I've wanted to have a "compatibility layer" that mimics my competitors APIs but have been scared of the possible repercussions.

This. Would love to hear more about this.
You can't copyright a softwarw API in the US because they are considered purely functional (in the mechanical sense). Take flathead screwdrivers as an example.

You can't copyright the dimensions of the blade head, since they're just the dimensions of the screw slot, and the only thing the copyright does is harm interoperability between screws and screwdrivers. You can copyright some other aspects of the screwdriver design, and if screwdrivers were a new thing, you could patent the idea of a screwdriver.

This is why the Java case hinged on a former sun employee literally cutting and pasting method implementations he wrote at Sun into Android's implementation while he worked at google. /double-face-palm

It wouldn't keep me up at night. Just do it.
We just added Wasabi as a destination option in Arq Backup. Seems to do that job well.
LeoFS: A stable and scalable S3 clone with NFS support?

That you can host on your own infrastructure

https://leo-project.net/leofs/

Well they edited the title so this isn't as tongue and cheek as it was meant to be. The original title was something like:

"Wasabi - a faster, better clone of S3?"

That's why I see others quote the original in the reply.

From the FAQ:

> 7. Your website indicates $.0039 per GB per month but the pricing comparison on the website indicates 1 TB is priced at $3.99 / month (instead of $3.90 / month for 1 TB). Why is that?

> The Wasabi monthly price is $.0039 GB / month. Given that there are 1024 GB in 1 TB (not 1000 GB), the price for 1 TB is $.0039 * 1024 or $3.99 per 1 TB per month.

Come on you are a digital storage company let's call things what they are. There are 1000 GB in a TB. There are 1024 GiB in a TiB.

If I am using Arq, will I have additional costs except '$.0039 per GB per month'?
The article states:

"Wasabi’s durability is 11 x 9s, the same as Amazon S3. To put that in context, if you stored 1 million 1 GB files in Wasabi, you would expect on average to lose one file every 659,000 years"

Can someone walk me through the math here? I specifically curious about why the size of the file being 1 GB is relevant to the calculation.

It doesn't matter. This is marketing fluff. What companies like Amazon do is count how many hard drives would need to simultaneously fail to lose data, calculate the odds of a hard drive losing data, and then multiplying. This gets you to 11 9's.

In practice, this is almost certainly not going to be why you lose data. That will be because of a chain of human errors, or a code bug, or because the user accidentally deleted the data, or because earthquakes destroyed your data centers.

I don't know anything about Wasabi other than what's on their web page, but I half suspect that what they did was look at Amazon's durability guarantee and then write that number down as their durability guarantee.

Yes, I think is disingenuous to call it "data" durability. "disk" durability would be more accurate.