Ask HN: What is the most economic and scalable way to host videos in 2022?
I'm wondering what the sweet spot is nowadays as I know a couple of startups delivering video content, and at a glance it seems this is costing them more to do than it should through the Paas options they are using. I would have thought this would have improved a lot in the 10-15 years since I last looked at it seriously. I'm hoping some here know how this ought to be done on modern cloud services to enable delivering in-app videos cost effectively to large numbers of viewers. thanks!
33 comments
[ 3.5 ms ] story [ 31.0 ms ] threadOr CloudFlare Stream, maybe: https://www.cloudflare.com/products/cloudflare-stream/
Edit to add: Please, for the love of god, do not self-host your video as a static file. There is a LOT of thought that goes into modern video streaming, such as on-the-fly transcoding to different resolutions & bitrates, pre-caching the next few seconds, seeking support, thumbnails, auto-captions, hardware decoder support... absolutely none of that is trivial to implement, but they make a huge difference in end-user UX. You may design your video on some sort of 5k supermac, but a lot of your viewers will be watching them on cheap Androids. The video providers spend a lot of time optimizing their streams and networks for such use cases. You will not, and your users will suffer for it, if you try to self-host. Please, please don't do this.
Signed, that guy who hates it when you're expected to download a 50 meg MP4 to understand a product.
If I follow these steps, what would I do wrong that would negatively affect my users:
1. Google 'ffmpeg hls nginx'
2. Encode my video at a bunch of different resolutions and bitrates.
3. Rent a server at hetzner or wherever.
4. Install nginx and do the minimal configuration needed to serve the files I prepared.
What could go wrong that would allow the video to play on a fast connection, but not on a slow connection (or slow device)?
Things that could go wrong: The end-user's frontend (a 4-year-old phone browser, a crappy smart TV, a Chromecast, Microsoft's browser abortions, whatever) has to support your protocol of choice. HLS is more and more common but it's not a guarantee, whereas the Vimeo and YouTube players have years of legacy and fallback support and have their CDNs pretty stabilized. With HLS you might still need a frontend JS player and then have to manage the distribution and packaging for that. And if you care about the video player UX, things like supporting user-chosen resolutions (overriding network speed), theater mode or easy full-screen or subtitles, 1.5x+ speed, etc. And you have to invalidate and re-CDN a bunch of separate files every time your source file updates.
But really, again, it's not that you CAN'T recreate what Vimeo and YouTube etc. have done, but why would you want to? It's a lot of effort to end up with an inferior UX, higher bandwidth charges, and a whole new pipeline (if not stack) that you have to maintain. No reason to reinvent the wheel for such a common use case.
Edit: And you STILL have to CDN it in the end, unless you have the phattest pipes in all the worlds. So you still have to subcontract with a third party in the end. Why not just put it there to begin with? It's silly.
It's one thing if you're Google-scale and need to in-house everything because you're dealing with a billion viewers a day, but when it's just a few thousand (the OP said that in a different comment), it's really not worth your time or bandwidth charges to reinvent that infrastructure.
But users wouldn't 'suffer' if I did self-host. They might not be able to play at double speed or whatever, but they'll still be able to play the file, even on an old device with a poor internet connection.
Some people have ideological reasons for wanting to self-host, and I wouldn't want to discourage them unnecessarily.
Off the top of my mind, there are only two web things I would strongly discourage people from self-managing whenever possible... video and HTTPS. Of course both are doable, and might be fun as a learning experience or a matter of ideological purity... but otherwise, their ROI is so, SO low it's not worth it (as in, you'll spend way more time and money fixing things than you would ever save having a pro do it).
And when you need wildcards, it gets even more complicated.
Obviously I'm not a very good sysadmin, but that's the point... I don't want to be and don't need to be, not when someone else can do it better and faster.
It's not an ideological thing, just a personal preference for making user-facing features as opposed to tinkering with the backend. Some people enjoy that sort of work. But users don't see it or care about it unless it breaks, which it often does in my experience as a mediocre full-stack dev with increasingly front-end leanings.
Edit: And on newer-generation serverless/Jamstack hosts (Cloudflare, Vercel, Netlify, etc.) you might never even get a shell to tinker with Certbot on. Which is arguably a good thing, especially compared to the bad old days of having to quadruple-configure HTTPS across your CDN, Varnish, Nginx/Apache, Certbot, and maybe cPanel or similar too. Yikes. Every single part of the intermediary chain was prone to frequent breakage. HTTPS these days isn't for authentication anyway, just for preventing drive-by MITM. Even EV certs aren't a good guarantee. So why do it yourself?
Just use Heroku, Netlify etc.
Simple cryptography was possible, although I didn't use it.
The only thing I really needed to hire was a CDN.
That way I didn't step anywhere near of a vendor lock-in trap, had a high degree of control over my platform and could even develop some on-premises customizations that would be simply impossible had I relied on third-party, internet-only solutions.
So I don't know. Of course I wouldn't reach the scale of yt or vimeo (and I didn't intend that anyway at all), but I guess the things you are talking about are not as hard to do as you think.
Unless somebody was bookmarking your video directly (does that really happen, as opposed to a HTML page / app), this IS what the video hosts offer, just at a higher level. Vimeo isn't a social video platform the way YouTube is, it's just a video host for the most part, including whitelabeling options. What is the vendor lock-in concern?
At the last educational place I worked for, we hosted our videos on YouTube, Vimeo, our own host, and a few other places. Nobody ever knew or asked which was which, except the YT and Vimeo videos were smoother experiences. They were all whitelabeled or else unlabeled.
I'd argue that for most content producers and users, YouTube (for the social network and the wide compatibility across devices) and Vimeo (for the ridiculously cheap pricing and ease of use) are the best choices AS CDNs. Both offer superior developer/producer experiences than a self-hosted stack too.
> had a high degree of control over my platform and could even develop some on-premises customizations that would be simply impossible had I relied on third-party, internet-only solutions.
But I'd love to hear about these cases. What justifies that complexity for you?
About the use cases and motivations, some of the drivers were commercial -- e.g. clients with content they considered too sensitive to be hosted off-premises; clients distributing content to places with limited connectivity; also, integration with custom access control systems.
The list goes on.
This. That's so many points separate points of failure, and doesn't that mean you have to get a dev involved (or have an in-house automated video pipeline) anytime someone wants to post or update a video? Maybe we just have different thresholds of "complexity". I try to keep clients away from unnecessarily bespoke tools, and keep them on simple GUIs supported by proper vendors... but that's just in my sphere of work. In bigger teams it might make sense to in-house more stuff.
Vimeo is a flat $20/mo, subject to fair use (approx 2 TB/mo). Worth doing the math to see which makes more sense for you.
We used Vimeo to serve an approx 30 sec video to about a million viewers a month and it was great. We noticed some minor downtime (few minutes at a time) but that was acceptable to us.
Cloudflare might be more reliable, not sure? Network infra is their bread and butter, and I tremendously admire them, but their streaming offering was overkill for our basic needs.
Ultimately their target markets are different. Cloudflare is an infrastructure company targeting devops teams, Vimeo targets small-to-medium content-driven businesses who don't want to do devops. Probably Vimeo's control panel is easier to use for content producers, whereas Cloudflare might have better APIs for devops types?
At a few thousand viewers a month, it's a pretty trivial use case and either company (or any other professional video host) would probably be just fine. It comes down to your own pipeline/production or business needs, probably.
As for being able to switch from one host to another, hopefully you're promoting a landing page of some sort and not a direct video URL. In that case you should be able to switch video providers at any time without interruption. Just keep the old version up alongside the new one for a few days, in case anyone has the page cached.
If you're lazy, just put it on YouTube as an unlisted video and embed it wherever. It'll be way more reliable (not to mention transcodable for different mobile viewers) than self-hosting on a web server.
At what scale? (Netflix or a small conference with 1000 users streaming?)
Do you have expertise? (Are you a kubernetes/nixos guru with extensive colocation administration experience or do you want Cloudflare Stream ease of use?)
The closest thing to an answer for the MOST economic and scalable way assumes a lot of devops capability: be your own CDN.
Hell, even if put your video on an established peer to peer network (similar to bittorrent streaming), the support nightmares would probably cost more than just putting it on an already established CDN.
There is really no reason to self-host videos these days. They are write-once (or very rarely), invalidated very rarely, and can be trivially cached everywhere, sort of a best-case scenario for big CDNs. Even if you didn't have to touch a single line of code, trying to reestablish just the business deals yourself across many countries and data interconnects would be ridiculously expensive.
FYI we use Vimeo to serve about a million views a month, for $20/mo. It doesn't get much cheaper than that.
If you need to be able to charge PPV or per month or whatever, Vimeo also offers a billable solution: https://vimeo.com/ott/pricing (OTT is apparently "over-the-top", as in PPV that bypasses cable/satellite/etc... according to wikipedia. Must be a cable-era term.) There are a bunch of other providers in this space... here is one list by a company: https://www.dacast.com/blog/how-does-white-label-video-strea...
Or you might be able to API-integrate Cloudflare Stream (or a similarly low-level CDN offering) with the LMS system directly, if they offer such interconnects.
My guess is that ease of use would probably get you a better ROI than strong piracy protections in this case and it's probably not worth it to go for DRM or rental models? IMHO only.
Here's one of Google's: https://support.google.com/displayvideo/answer/6086451?hl=en
Apart from that, ffmpeg can generate all the files you need and nginx can host them, but make sure you have a hosting company with generous traffic allowance. Putting this on S3 is probably a lot more expensive than renting vimeo, which is a dedicated white label video hosting service.