HN Explorer

Ask HN: Does anyone need a cheap way to move data OUT of Amazon S3?

6 points by ryanworl ↗ HN
Hi HN,

I've recently come up with a method retrieving data out of S3 indirectly using another AWS service for less than the posted egress rates. It is not an exploit of a security vulnerability or anything remotely similar. Also doesn't rely on compression, nor does it use the AWS Export services like Snowball. Technically this could be applied to any data within AWS, not just S3.

I can do this for less than 2 or 3 cents per GB, potentially less, and reliably. I have benchmarked this technique with some data of my own moving files from S3 to Google Cloud Storage.

Does anyone have a use to hire me to implement this technique for them, or know anyone who does? It would be a significant savings over the standard egress rates for people needing to transfer many terabytes of data.

Email is in my profile.

8 comments

[ 3.0 ms ] story [ 31.1 ms ] thread
nik736
Lightsail?
ryanworl
No, this isn't related to Lightsail. It may be worth investigating though! I haven't looked at all their bandwidth pricing rules.

EDIT: It looks like there is a hard limit of 20 Lightsail instances per account which would provide a limited amount of bandwidth. The docs and the website conflict so I can't confirm that for sure. Either way, not related to my technique.

[dead] num2499
ได้ เครียร์รอมไห้ด้วย
QuinnyPig
Interesting. Does this scale / act responsively enough to serve data to the public, or is it restricted to bulk transfers?
ryanworl
In theory not impossible to serve a request with this, but the performance would be poor in terms of TTFB. More likely useful in bulk transfers.
[deleted]
(comment deleted)
posnet
Does this get around the s3 request costs? or is this just a cheaper way to account far large data volumes.
ryanworl
It still requires regular requests to S3, so large objects would see the most savings.