HN Explorer

Ask HN: Clever ways to run a media-heavy website on a budget?

181 points by wild_preference ↗ HN
I've been taking a break from working to relax and visit some friends in a few countries. So I don't have a lot of money, but with my newfound free time, I've been inspired to take a shot at a few ideas I've always wanted to build.

One idea is particularly media heavy. I have 5+ TB of media that I'd like to build something around.

Services like S3 are pretty expensive, especially bandwidth. Backblaze has been trying to market their storage solution (https://www.backblaze.com/b2/cloud-storage-pricing.html) against S3.

But I can get even cheaper by using low quality hardware and bandwidth like Kimsufi's 2TB machines for $10/mo (https://www.kimsufi.com/en/servers.xml).

Does anyone have another ideas for stretching a dollar?

98 comments

[ 3.5 ms ] story [ 184 ms ] thread
megamindbrian2
You could transcode with VLC/Elastic Transcoder and store that in the cloud until an individual file is requested.
hunvreus
Cloudflare is free: CDN the heck out of it.
scrollaway
This very much. You can also save all your media with content hashes in their filename, then tell cloudflare to "always permanently cache" everything behind a certain path using a page rule.

I have a site set up like that which serves over 1TB monthly and <5GB of it actually goes through to S3. (And obviously you don't have to use s3 to back it, you can use anything)

Bonus points: If you do it this way you can also use immutable caching for clientside savings. More info: https://hacks.mozilla.org/2017/01/using-immutable-caching-to...

This may be dependent on how large your media is however. 5TB sounds like it could be large videos, so some of this may not apply to you. I don't know how well Cloudflare caches videos.

SXX
Any evidence that Cloudflare will cache even fraction of his 5TB on free plan? While I personally have sites where like 20GB of images were cached on free tier I truly don't expect them to cache as much as even 1TB.

> I don't know how well Cloudflare caches videos.

It's cache files only under 512MB:

https://support.cloudflare.com/hc/en-us/articles/200394750-W...

vidyesh
Does Cloudflare cache everything no matter how much you feed it? You said <5GB goes through to S3 so that is from the total 1TB of data?
joshribakoff
They have a history of randomly deciding it's not free and shutting off people's service without notice. It also slows down your site (cache misses)
sudhirj
Not so randomly - they seem to shut down sites that take the CDN freeloading to an extreme - image hotlinks, lots of data with not HTML, that kind of thing. If the entire site is being served via CF, they might not mind. They do have a video service in the works, though, need to see how that pans out.
joshribakoff
What about a site with a little HTML but lots of files. "Might not mind" doesn't sound like something I would bet my business on. There is a gray area and you can find examples of people that got shut down (without warning) whom would disagree that they crossed that line
sudhirj
Get a paid plan if you’re a business or are serious about your site. Don’t think they take unilateral action on paid plans. There’s also an option to pay for bandwidth somewhere, there certainly is when you activate Argo. Can’t really accuse you of free loading once you pay.
pecicek
In case Cloudflare's free plan is not enough, I'd suggest going for proven low cost providers such as CDN77, KeyCDN or MaxCDN.
DDR0
How about using something like Websockets to reduce outbound cost? It'll have anyone on the page viewing the media also host the media for their fellow web surfers. If I recall, there is a socket-based bittorrent library floating around somewhere - https://github.com/webtorrent might be a good start. :)

Serve the static web-page files from a decent host. Seed the 5tb of content from the cheap host.

voltagex_
And what if your users are charged for upstream bandwidth?
solarkraft
That's

a. Unlikely, where have you seen this before?

b. A reasonable price to view the page.

AzMoo_
Uploads are metered on loads of Internet plans in Australia.
voltagex_
I was thinking more of phone data plans - they're even more limited (although strangely Australia's got some of the best plans at the moment outside of Europe)
argestes
Mobile internet plans mostly charge for upstream too. (I'm from Turkey)
fratlas
1TB for $5/month on digitalocean ain't bad
wolco
That's only 25g of storage
loganekz
s3 with cloudfront

https://docs.aws.amazon.com/AmazonCloudFront/latest/Develope...

lubos
That's probably the most expensive route you can go once you have any decent traffic.
th0ma5
What about text content only? This is free SSL with Let's Encrypt but Cloudfare seemingly charges for SSL.

Edit: Seemingly it could be free, but you must manually handle your certs to some degree.

Edit 2: Cloudfront now comes with "AWS Shield Standard" automatically to protect from DDOS?

wild_preference
I've actually used that combination before and I would hate for the person that DDoSed me to know how much they cost me.
CodeWriter23
Definitely hide whatever you implement behind CloudFlare.
kondro
I think literally the only more expensive option you could’ve suggested is Akamai.
Geee
What kind of media? You might be able to avoid hosting them on your own servers. If it's video, you can use Youtube. If it's images, use Flickr or something else.
zawerf
I don't recommend this, but if your media are images, some really large manga sites (kissmanga.com, a top 500 site in the US according to alexa) have been using blogger.com to host from Google Album Archive for years now and still haven't been shut down. They are probably pushing hundreds of terabytes in bandwidth per month for free.
alex621101
Linode $5 server + cloudflare for free
icebraining
A Linode $5 server only has 20GB of storage. OP has 5TB.
kondro
Cloudflare question: when would they start to get mad at you using potentially PBs of bandwidth per month on a free plan?
jgrahamc
It depends what you're doing. If you use us as some kind of file host then that's a ToS violation.
kondro
Those TOS are both very restrictive and vague at the same time though.

On one hand specifying that literally only HTML can be hosed through the platform whilst simulataneously suggesting that caching other content is fine, as long as it’s not “disproportionate” (but disproportionate to what?).

“SECTION 10: LIMITATION ON NON-HTML CACHING You acknowledge that Cloudflare’s Service is offered as a platform to cache and serve web pages and websites and is not offered for other purposes, such as remote storage. Accordingly, you understand and agree to use the Service solely for the purpose of hosting and serving web pages as viewed through a web browser or other application and the Hypertext Markup Language (HTML) protocol or other equivalent technology. Cloudflare’s Service is also a shared web caching service, which means a number of customers’ websites are cached from the same server. To ensure that Cloudflare’s Service is reliable and available for the greatest number of users, a customer’s usage cannot adversely affect the performance of other customers’ sites. Additionally, the purpose of Cloudflare’s Service is to proxy web content, not store data. Using an account primarily as an online storage space, including the storage or caching of a disproportionate percentage of pictures, movies, audio files, or other non-HTML content, is prohibited. You further agree that if, at Cloudflare’s sole discretion, you are deemed to have violated this section, or if Cloudflare, in its sole discretion, deems it necessary due to excessive burden or potential adverse impact on Cloudflare’s systems, potential adverse impact on other users, server processing power, server memory, abuse controls, or other reasons, Cloudflare may suspend or terminate your account without notice to or liability to you

pornel
No. If it's just lots of files & lots of regular users, and you're not running anything illegal/abusive, then it's fine.
berbec
GSuite + rclone mount for storage + unlimited bandwidth vps. Check out lowendbox.com
indigodaddy
Check out BuyVM.net's Storage VPS offerings. Unlimited transfer apparently. Ultra competitive pricing and they don't overprovision. They're well established and quite reputable in the hosting community. Largest disk space offering appears to be 2TB, however if you contact them they might be able to work something up for you.

Note: No affillation, but I've had a few small VPS'es with the. Very reliable shop.

phasecode
Store your files in backblaze then proxy them through some cheap vps (to cover bandwidth) and add local cache through something like nginx so you're not going to the source frequently.
iMerNibor
Some hints: ovh is good for eu/(northern) us (loadbalancing ip + vps behind it to spread load - be prepared to ditch vps if you're getting horrible performance), scaleway if you can live with just eu Probably as cheap as it gets from big providers

Obviously going to have to compromise (round robin dns > ~1gbps, self managed stuff), but thats just expected If it's just images probably just go with cloudflare

joshribakoff
Can make byte range requests a bit complicated and slows down your site when there's cache misses
fleitz
Use the slice command in nginx and $slice_range your cache key ;)
joshribakoff
I'm not asking for advice I'm stating it does technically complicate your setup. What if two people request overlapping ranges concurrently etc... it is more moving parts to setup, test and administer
pnutjam
That's 9.99 Euro's. I use Time4vps.eu. Their prices are the same, but I have an older plan that was cheaper. They do offer specials occasional as well. I have their 1TB plan for about $15 /quarter.
dawnerd
I have a server from here: https://www.nocix.net/dedicated/

Not necessarily the most reliable provider but hard to beat the prices, especially for their larger servers.

Since you said you were working with media, a vps will probably have pretty poor performance if you're doing any transcoding or the such.

mappu
Wasabi (S3-compatible) recently released an "Unlimited Egress" plan with unmetered bandwidth: https://wasabi.com/pricing/

That puts 5TB storage at $24/mo.

jazoom
Do you have any idea where their datacentres are?

They claim they're 6x faster than S3 but I'll bet it's 20 times slower with latency unless they have a Sydney datacentre.

You'd think this is info they'd put somewhere findable on their website.

voltagex_
"Wasabi is deployed in fully secure and redundant data centers that are certified for SOC-2, ISO 27001, and PCI-DSS. Our primary production data center is in the us-east region and additional data centers in other regions will be activated soon. Please contact us if you have specific questions in this area."
stordoff
Worth noting that:

> If your use case creates an unreasonable burden on our infrastructure, we reserve the right to limit your egress traffic and/or ask you to switch to our Legacy pricing plan. Wasabi’s hot cloud storage service is not designed to be used to serve up (for example) web pages at a rate where the downloaded data far exceeds the stored data or any other use case where a small amount of data is served up a large amount of times.

https://wasabi.com/pricing/pricing-faqs/

codegeek
Look at DigitalOcean spaces. 250 GB storage, 5TB bandwidth for $5/Month. Works with most S3 compatible sdks already
[deleted]
(comment deleted)
joshribakoff
Digital ocean has super cheap block storage and currently has no way to track bandwidth usage (currently overages are not billed). GlusterFS is also useful. I cut latency in half switching from s3 to glusterFS
CodeWriter23
I believe you mean super cheap Object storage. Their block storage is $0.10/GB/Mo. which is pretty much the going rate.
SXX
One of cheapest options with 20TB+ of free bandwidth would be Hetzner auction. These servers might be not super reliable, but they don't have setup fee and still capable of serving your files:

https://robot.your-server.de/order/market

tudorconstantin
I am using hetzner for about 5 years and had absolutely no problem with them.

I am paying 30 EUR/month for a machine with an older generation i7 CPU, 24 GB of RAM, 2x750 GB HDD and 20TB of monthly traffic. The price is close to the electricity cost if I were to host the machine in my home. During these 5 years the machine had not a single downtime.

I've read stories that they had lousy tech support, but can't confirm, since I never needed it.

Overall I had a great experience with them so far, so I also recommend them whenever I can.

zorked
Another data point: I used to buy servers in Hetzner's auction by the dozens per month. It's very common that the servers will crash and need new HDDs or RAM. I never had a problem with support, but soon learned to treat those servers like cattle: if one is broken, return it and grab a new one.

They are worth it even if you have trouble, though. Just make sure you have a backup plan.

tehlike
What is your bw requirement?
rocky1138
Your description doesn't include enough information. My first thoughts were to create a torrent.
chx
Write up more precisely of how much bandwidth, storage and CPU you need and post to https://www.lowendtalk.com/
voltagex_
Has anyone successfully run a business from an OpenVZ VPS? I thought those ones, especially the ones on LowEndTalk were massively oversubscribed.
chx
1. You get what you pay for 2. Lowendtalk has a wide spectrum of offers, OpenVZ, KVM, dedis and more.
mozumder
I use my home ISP with a home server on a static IP address.

My server is something I built myself, and contains about 30TB of media storage (FreeBSD ZFS setup), plus NVMe SSDs for database.

Unlimited bandwidth, unlimited storage, and super fast.

Holy crap is it fast.

icebraining
What's your upload bandwidth? Around here they're 1/10 of the download, so you'd need a pretty expensive plan to serve a heavy website.
mozumder
Same upload & download. This is on Verizon Business FIOS.
rajacombinator
If you’re hosting something with enough bandwidth costs to be expensive, you should probably find a business model and monetize it.
mkj
Could you use better compression?