Can anyone explain specifically what the YT code does that the existing python interpreter is unusable and apparently quickjs takes 20 minutes to run it?
Is it just a lot of CPU-bound code and the modern JIT runtimes are simply that much faster, or is it doing some trickery that deno optimizes well?
I was surprised they went with Deno instead of Node, but since Deno has a readily available single-exe distribution that removes a lot of potential pain. This was pretty much just a matter of time, though; the original interpreter in Python was a brilliant hack but limited in capability. It was discussed a few years ago for the YouTube-dl project here https://news.ycombinator.com/item?id=32793061
Keep in mind that yt-dlp doesn't just support YouTube, which-- notwithstanding the claims of "all DRM is malware" etc.-- probably won't download actively harmful code to your computer: it also supports a huge number of video streaming sites, including some fairly obscure and sketchy ones. Sandboxing in the interpreter that's at least as good as what you'd get in a browser is a must, because by design this is doing untrusted code execution.
I’m a paying YouTube premium subscriber. Last weekend, I wanted to download something so I can watch it on my way in the train. The app got stuck at “waiting for download..” on my iPad. Same on iPhone. Restart did not work. I gave up after an hour (30 mins hands on trying stuff, 30 mins waiting for it to fix itself). Downloaded the video using yt-dlp, transferred it to my USB c flash drive, and watched it from that.
Awaiting their “premium cannot be shared with people outside household” policy so I can finally cancel. Family members make good use of ad-free.
I also have YouTube premium and watch mostly on my iPad and TV. YouTube constantly logs me out at least once per day. I notice because I’ll randomly start seeing ads again (I open videos from my rss reader, never their site). This never happened when I wasn’t on premium. I don’t get what they’re doing, but my impression after almost a year is that it’s only slightly less annoying than getting ads. At this point, I might as well not renew and just use ad block.
So long as they are broadcasting media to the public without an explicit login system, so as to take advantage of public access for exposure, it will remain perfectly legitimate and ethical to access the content through whatever browser or software you want.
After they blitzed me with ads and started arbitrarily changing features and degrading the experience, I stopped paying them and went for the free and adblocking clients and experience.
I may get rid of phones from my life entirely if they follow through with blocking third party apps and locking things down.
I'm constantly baffled by how bad the implementation of YouTube Premium downloads is. Videos will buffer to 100% in a matter of seconds but get endlessly stuck when I hit the download button. Why? All the bytes are literally on my device already.
I run into that download issue all the time. I need to pause downloading each video. Force close the youtube app. Then unpause the downloads to get them downloading again. It has been happening for years and is still unfixed.
Also a paying YT Premium subscriber. I live in a rural part of CA where there isn't much 5G reception. For extremely long drives in my minivan, I allow my toddler to watch Ms. Rachel on the screen via an HDMI port input from my iPhone. Youtube Premium videos have DRM that disallow downloads to play over HDMI, so I had to do what you did and add them as files locally to VLC and play them from there.
I'll admit to using yt-dlp to get copies of videos I wish to have a local copy of, which can't be taken away from me by somebody else, but I pay for premium because that pays for content I watch. If you don't pay for content, where's it going to come from? Patreon only works for super dedicated stars with a huge following.
I have the opposite problem... frequently streaming a video gets stuck buffering even on my gigabit fiber connection, but I can download a full max quality version in a matter of seconds.
Looks like this runtime is written in Rust. Really does seem like Rust is rapidly swallowing all kinds of common tools and libraries. In this case a single compiled binary for multiple architectures is quite convenient for something like yt-dlp.
I wonder how long until it gets split off into its own project. For the time being, it could do with a lot more documentation. At least they've got some tests for it!
I used to work on video generation models and was shocked at how hard it was to find any videos online that were not hosted on YouTube, and YouTube has made it impossibly hard to download more than a few videos at a time.
> YouTube has made it impossibly hard to download more than a few videos at a time
I wonder why. Perhaps because people use bots to mass-crawl contents from youtube to train their AI. And Youtube prioritizes normal users who only watch a few videos at most at the same time, over those crawling bots.
Nsig/sig - Special tokens which must be passed to API calls, generated by code in base.js (player code). This is what has broken for yt-dlp and other third party clients. Instead of extracting the code that generates those tokens (eg using regular expressions) like we used to, we now need to run the whole base.js player code to get these tokens because the code is spread out all over the player code.
PoToken - Proof of origin token which Google has lately been enforcing for all clients, or video requests will fail with a 403. On android it uses DroidGuard, for IOS, it uses built in app integrity apis. For the web it requires that you run a snippet of javascript code (the challenge) in the browser to prove that you are not a bot. Previously, you needed an external tool to generate these PoTokens but with the Deno change yt-dlp should be capable of producing these tokens by itself in the near future.
SABR - Server side adaptive bitrate streaming, used alongside Google's UMP protocol to allow the server to have more control over buffering, given data from the client about the current playback position, buffered ranges, and more. This technology is also used to do server-side ad injection. Work is still being done to make 3rd party clients work with this technology (sometimes works, sometimes doesn't).
And barely a few days after google did it the fix is in.
Amazing how they simply couldn't win - you deliver content to client, the content goes to the client. Could be the largest corporation of the world and we still have yt-dlp.
That's why all of them wanted proprietary walled gardens where they would be able to control the client too - so you get to watch the ads or pay up.
This will be interesting to see how it affects the numerous Android apps on F-Droid that are essentially wrappers around yt-dlp to create a YouTube Music clone.
At some point we’re going to need a better place to put videos than YouTube. The lack of any democratization of bulk storage is beginning to be a real problem on the internet.
Yes, we have archive.org. We need more than that, though.
I’m sure there’s some distributed solution like IPFS but I haven’t seen any serious attempt to make this accessible to every day people.
The problem with bulk storage is that it will be abused at large scale.
CSAM peddlers, intellectual property violators, unconsensual sexual material ("revenge porn"), malware authors looking for places to exfiltrate stolen data, propagandists and terrorists, the list of abusers is as long as it is dire.
And for some of these abuser classes, the risk for any storage service is high. Various jurisdictions require extremely fast and thorough responses for a service provider to not be held liable, sometimes with turnaround times of 24 hours or less (EU anti terrorism legislation), sometimes with extremely steep fines including prison time for responsible persons. Hell, TOR exit node providers have had their homes raided and themselves held in police arrest or, worse, facing criminal prosecution and prison time particularly for CSAM charges - and these are transit providers, not persistent storage.
And all of that's before looking on the infrastructure provider side. Some will just cut you off when you're facing a DDoS attack, some will bring in extortionate fees (looking at you, AWS/GCE/Azure) for traffic that may leave you in personal bankruptcy. And if you are willing to take that risk, you'll still run the challenge of paying for the hardware itself - storage isn't cheap, 20TB of storage will be around 200€ and you want some redundancy and backups, so the actual cost will rather be 60-100€/TB plus the ongoing cost of electricity and connectivity.
That's why you're not seeing much in terms of democratization.
The writing is on the wall for easy ripping. If there's any YT content you expect you'll want to preserve for a long time, I suggest spinning up https://www.tubearchivist.com/ or something similar and archiving it now while you still can.
That makes it seem like yt-dlp itself was rewritten from Python to JavaScript (for those who even know it’s Python) or that it used to use Node and now uses Deno.
What I found much more annoying, and so far have not been able to work around, is that yt-dlp requires you to have a YouTube account, something that I have not had for a decade or so, and am unwilling to create again.
What tool can I use to simply store what my browser receives anyway, in a single video file?
yt-dlp has never required an account. If it looks like that, either you're seeing some error from, e.g., youtube and not yt-dlp claiming that or you're running some sort of scam version instead of the real thing.
Days of just getting data off the web are coming to an end as everything requires a full browser running thousands of lines of obfuscated js code now. So instead of a website giving me that 1kb json that could be cached now I start a full browser stack and transmit 10 megabytes through 100 requests, messing up your analytics and security profile and everyone's a loser. Yay.
On the bright side, that opens an opportunity for 10,000 companies whose only activity is scraping 10MB worth of garbage and providing a sane API for it.
Luckily all that is becoming a non-issue, as most content on these websites isn't worth scraping anymore.
> Days of just getting data off the web are coming to an end
All thanks to great ideas like downloading the whole internet and feeding it into slop-producing machines fueling global warming in an attempt to make said internet obsolete and prop up an industry bubble.
The future of the internet is, at best, bleak. Forget about openness. Paywalls, authwalls, captchas and verification cans are here to stay.
This 1kb os json still sounds like a modern thing, where you need to download many MB of JavaScript code to execute and display the 1kb json data.
What you want is to just download the 10-20kb html file, maybe a corresponding css file, and any images referenced by the html. Then if you want the video you just get the video file direct.
Simple and effective, unless you have something to sell.
I think this is just another indication of how the web is a fragile equilibrium in a very adversarial ecosystem. And to some extent, things like yt-dlp and adblocking only work if they're "underground". Once they become popular - or there's a commercial incentive, like AI training - there ends up being a response.
It's an arms race. Websites have become stupidly/unnecessarily/hostilely complicated, but AI/LLMs have made it possible (though more expensive) to get whatever useful information exists out of them.
Soon, LLMs will be able to complete any Captcha a human can within reasonable time. When that happens, the "analog hole" may be open permanently. If you can point a camera and a microphone at it, the AI will be able to make better sense of it than a person.
fortunately it is now easier than ever to do small-scale scraping, the kind yt-dlp does.
I can literally just go write a script that uses headless firefox + mitmproxy in about an hour or two of fiddling, and as long as I then don't go try to run it from 100 VPS's and scrape their entire website in a huge blast, I can typically archive whatever content I actually care about. Basically no matter what protection mechanisms they have in place. Cloudflare won't detect a headless firefox at low (and by "low" I mean basically anything you could do off your laptop from your home IP) rates, modern browser scripting is extremely easy, so you can often scrape things with mild single-person effort even if the site is an SPA with tons of dynamic JS. And obviously at low scale you can just solve captchas yourself.
I recently wrote a scraper script that just sent me a discord ping whenever it ran into a captcha, and i'd just go look at my laptop and fix it, and then let it keep scraping. I was archiving a comic I paid for but was in a walled-garden app that obviously didn't want you to even THINK of controlling the data you paid for.
Fortunately the community is not alone in this fight, because many AI companies need to be able to download YT videos. But they should sponsor yt-dlp more directly..
jdownloader2 is one of those weirdly licensed things, it claims to be "open source" but you actually can't find the entire source anywhere. Worse than proprietary, tbh.
More specifically, yt-dlp uses legacy API features supported for older smart TVs which don't receive software updates. Eventually once that traffic drops to near zero those features will go away.
That conspiracy theory never even made sense to me. Why would anyone think that a payment and ad-supported content platform secretly wants their content to be leaked through ad and payment free means?
141 comments
[ 3.5 ms ] story [ 103 ms ] threadIs it just a lot of CPU-bound code and the modern JIT runtimes are simply that much faster, or is it doing some trickery that deno optimizes well?
Awaiting their “premium cannot be shared with people outside household” policy so I can finally cancel. Family members make good use of ad-free.
1. Unlimited YouTube Premium
2. Unlimited drink reimbursement (coffee, tea, smoothies, whatever)
The psychological sense of loss from those two things would be larger than any 5% raise.
I recently got paused for "watching on another device" when I wasn't. I don't think that policy you mention is too far off.
So long as they are broadcasting media to the public without an explicit login system, so as to take advantage of public access for exposure, it will remain perfectly legitimate and ethical to access the content through whatever browser or software you want.
After they blitzed me with ads and started arbitrarily changing features and degrading the experience, I stopped paying them and went for the free and adblocking clients and experience.
I may get rid of phones from my life entirely if they follow through with blocking third party apps and locking things down.
https://github.com/yt-dlp/yt-dlp/blob/2025.09.23/yt_dlp/jsin...
I wonder why. Perhaps because people use bots to mass-crawl contents from youtube to train their AI. And Youtube prioritizes normal users who only watch a few videos at most at the same time, over those crawling bots.
Who knows?
PoToken - Proof of origin token which Google has lately been enforcing for all clients, or video requests will fail with a 403. On android it uses DroidGuard, for IOS, it uses built in app integrity apis. For the web it requires that you run a snippet of javascript code (the challenge) in the browser to prove that you are not a bot. Previously, you needed an external tool to generate these PoTokens but with the Deno change yt-dlp should be capable of producing these tokens by itself in the near future.
SABR - Server side adaptive bitrate streaming, used alongside Google's UMP protocol to allow the server to have more control over buffering, given data from the client about the current playback position, buffered ranges, and more. This technology is also used to do server-side ad injection. Work is still being done to make 3rd party clients work with this technology (sometimes works, sometimes doesn't).
Nsig/sig extraction example:
- https://github.com/yt-dlp/yt-dlp/blob/4429fd0450a3fbd5e89573...
- https://github.com/yt-dlp/yt-dlp/blob/4429fd0450a3fbd5e89573...
PoToken generation:
- https://github.com/yt-dlp/yt-dlp/wiki/PO-Token-Guide
- https://github.com/LuanRT/BgUtils
SABR:
- https://github.com/LuanRT/googlevideo
EDIT2: Addeded more links to specific code examples/guides
Amazing how they simply couldn't win - you deliver content to client, the content goes to the client. Could be the largest corporation of the world and we still have yt-dlp.
That's why all of them wanted proprietary walled gardens where they would be able to control the client too - so you get to watch the ads or pay up.
Yes, we have archive.org. We need more than that, though.
I’m sure there’s some distributed solution like IPFS but I haven’t seen any serious attempt to make this accessible to every day people.
Almost 25 years on the internet and I have not been able to download anything from IPFS. Does one need a PhD to do so?
CSAM peddlers, intellectual property violators, unconsensual sexual material ("revenge porn"), malware authors looking for places to exfiltrate stolen data, propagandists and terrorists, the list of abusers is as long as it is dire.
And for some of these abuser classes, the risk for any storage service is high. Various jurisdictions require extremely fast and thorough responses for a service provider to not be held liable, sometimes with turnaround times of 24 hours or less (EU anti terrorism legislation), sometimes with extremely steep fines including prison time for responsible persons. Hell, TOR exit node providers have had their homes raided and themselves held in police arrest or, worse, facing criminal prosecution and prison time particularly for CSAM charges - and these are transit providers, not persistent storage.
And all of that's before looking on the infrastructure provider side. Some will just cut you off when you're facing a DDoS attack, some will bring in extortionate fees (looking at you, AWS/GCE/Azure) for traffic that may leave you in personal bankruptcy. And if you are willing to take that risk, you'll still run the challenge of paying for the hardware itself - storage isn't cheap, 20TB of storage will be around 200€ and you want some redundancy and backups, so the actual cost will rather be 60-100€/TB plus the ongoing cost of electricity and connectivity.
That's why you're not seeing much in terms of democratization.
"yt-dlp moves to Deno runtime"
That makes it seem like yt-dlp itself was rewritten from Python to JavaScript (for those who even know it’s Python) or that it used to use Node and now uses Deno.
What tool can I use to simply store what my browser receives anyway, in a single video file?
I want them to go overboard. I want BigTech to go nuts on this stuff. I want broken systems and nonsense.
Because that’s the only way we’re going to get anything better.
Luckily all that is becoming a non-issue, as most content on these websites isn't worth scraping anymore.
All thanks to great ideas like downloading the whole internet and feeding it into slop-producing machines fueling global warming in an attempt to make said internet obsolete and prop up an industry bubble.
The future of the internet is, at best, bleak. Forget about openness. Paywalls, authwalls, captchas and verification cans are here to stay.
What you want is to just download the 10-20kb html file, maybe a corresponding css file, and any images referenced by the html. Then if you want the video you just get the video file direct.
Simple and effective, unless you have something to sell.
* PeerTube and similar platforms for video streaming of freely-distributable content;
* BitTorrent-based mechanisms for sharing large files (or similar protocols).
Will this be inconvenient? At first, somewhat. But I am led to believe that in the second category one can already achieve a decent experience.
Soon, LLMs will be able to complete any Captcha a human can within reasonable time. When that happens, the "analog hole" may be open permanently. If you can point a camera and a microphone at it, the AI will be able to make better sense of it than a person.
I can literally just go write a script that uses headless firefox + mitmproxy in about an hour or two of fiddling, and as long as I then don't go try to run it from 100 VPS's and scrape their entire website in a huge blast, I can typically archive whatever content I actually care about. Basically no matter what protection mechanisms they have in place. Cloudflare won't detect a headless firefox at low (and by "low" I mean basically anything you could do off your laptop from your home IP) rates, modern browser scripting is extremely easy, so you can often scrape things with mild single-person effort even if the site is an SPA with tons of dynamic JS. And obviously at low scale you can just solve captchas yourself.
I recently wrote a scraper script that just sent me a discord ping whenever it ran into a captcha, and i'd just go look at my laptop and fix it, and then let it keep scraping. I was archiving a comic I paid for but was in a walled-garden app that obviously didn't want you to even THINK of controlling the data you paid for.
svn://svn.jdownloader.org/jdownloader/trunk
it's right there on the home page
It's it's always been very apparent that YouTube are doing _just enough_ to stop downloads while also supporting a global audience of 3 billion users.
If the world all had modern iPhones or Android devices you'd bet they'd straight up DRM all content
[0] https://windowsread.me/p/best-youtube-downloaders
[1] https://news.ycombinator.com/item?id=45300810