I second that. But yes you are right it's my fault that I haven't done that already. Be assured that I will include some information regarding what you said in the readme.
It’s odd to me how nontechnical people have migrated to GitHub to make bug reports on software they use.
The other day, I was trying to use a Python library to scrape some stuff. It kept breaking. I looked on the issues page and there were many, essentially useless bug reports asking them to fix the issue. I opened up the script and changed about 7 lines and it worked fine.
There are two factors at play here I think. The first is that Github has become quite popular. The second is that Github has worked a lot on usability - it's now very easy for users who are not very technical to create an account and report issues.
At times I imagine there could be some kind of "I know how to report bugs"-certification that people could somehow earn. Projects could then use this as a barrier to prevent getting too many useless issue reports.
The problem really goes both ways - as a software dev I'm disincentivized to report detailed bugs to projects, because there's a high risk it will get drowned out in the noise.
Some projects have quite specific templates for reporting issues. This helps a bit, but you still need someone to go in and remove issues that don't match the template, and few people want to volunteer for that work.
Yeah, I guess it’s just an entirely different kind of engagement.
I mostly use GitHub to find off the shelf solutions to whatever problem I’m having or to avoid doing the work myself. If I run into any problems I have with something from there, I either try and fix it by tweaking the repository, using another, or just building the thing myself. I think the last time I reported an issue on an external repository was because of something I was made to use at work.
I think a lot of it is maybe bleed over from gaming communities, especially emulators, maybe rom hacks too? I’ve noticed barrages of useless bug reports on those repositories. But I’ve been surprised to find them on far afield stuff as well.
At least there is a feature called 'discussions' coming soon to github. That should hopefully minimize the abuse of issues. But that is also due to the fact of current github. Not everyone is displaying their email and there is no other communication channel.
Being able to edit people’s comments on GitHub is nice as a maintainer but removing information and completely changing people’s comments is really nasty. Any such effort should be clearly justified, preferably with a visual “edit message” that goes along with the smaller “edited” text.
I don't think that editing comments that you didn't make should ever be a feature. This is something that could easily be abused to make it appear that someone else said something they didn't actually way.
I can see what you're saying in principle, but what if a bug is opened with "this stopped working for me" but evolves into a very specific issue? Should the maintainer open a new bug, link to it and close the old one?
In practice, I find it really useful to tidy up the top message of a bug report (but /really/ like seeing history of it). I also see cases where users spam raw logs into updates and tidying it up (perhaps making the log portion an attachment or changing formatting) makes it much clearer for future-you and other collaborators.
Counter-anecdote: Even YouTube support is somewhat flaky (age-restricted videos, private (as in, locked behind auth) videos, random failures), and several sites I use are broken and require additional options (cookies) for even basic functionality.
I use youtube-dl to download from Nico Nico Douga, which means having to restart the download every few minutes because the program doesn't send heartbeat requests. There's been a PR to fix this for a couple of years now [1], and a more recent updated PR to add encrypted video support [2], both of which are still open. Meanwhile, the corresponding PR on youtube-dlc [3] is actively being looked at.
It makes sense that it's easier to give <100 PRs attention than 25k+. If the DLC project grows, you can't know if the same won't happen again.
Not saying forks should never happen - streamlink and KeepassXC are examples of great forks off the top of my mind - but you can't judge just on anecdotal stale PRs.
> It makes sense that it's easier to give <100 PRs attention than 25k+.
ydl has 700 open PRs, not 25k+. ydl hasn't had 25k+ PRs in its lifetime either, it's had 4k: GH issues and PRs are the same object, so the sequence is shared.
Doesn't change the point. All of the fork's 55 PRs except 2
and most of the issues were made by the forker himself as well.
It's not like the forker inherited all the open issues/PRs, they just dropped them all.
Getting popular and having the masses leave a bunch of issues/PRs is hard. Reviewing code is hard and takes more expertise than writing it. A lot harder than reviewing your own PRs on a motivated (for now) fork.
> It's not like the forker inherited all the open issues/PRs, they just dropped them all.
That's wrong, most of the recent commits in the fork are merging branches from a lot of other repos. GitHub doesn't have a mechanism for inheriting a project's pull requests when you fork, but that doesn't mean they're automatically "dropped", since you can still merge the corresponding branch.
Actually I merged quite some PRs that were open or even closed. However I tested and reviewed all I can but those geoblocked ones. But most of these were verified already or I could use some simple self hosted vpn.
But do note that my reviews are not really in line with the strict coding conventions originating from the main project. As long as it works and isn't totally gibberish it will be approved by me. I rather want functionality and somewhat stability quite fast.
Although totally agree reviewing can be such a pain..
As someone who has tried to contribute to YouTube-dl in the past, the absolute disinterest I received and complete lack of follow up on simple bug-fixes caused me to completely dismiss YouTube-dl as a FOSS project worthy of my time.
And I say that as someone who sends PRs to projects I’ve visited once, over tiny things like typos or lack of installation/build instructions in the readme.
Some of my PRs received a crazy all-or-nothing treatment, asking me to fix huge, long-standing bugs nobody had fixed for years, instead of accepting a 2-line diff which did a provider-specific workaround which solved a real bug, but seemingly not in the “right” way.
Completely toxic and hostile.
That someone has started a community-fork with a more welcoming and pragmatic approach does not surprise me in the least.
I had basically the same experience. I even got requests for functionality that was already there, as if the maintainers didn't even read or test the patch. There are a number of pull requests of mine that have been waiting open for years, ready to be merged.
There is an issue preventing the download of Watch Later playlists that has been open for over a year. The core maintainer, dstftw, closes all reports related to it with "duplicate" while never giving a reason or the original issue. See this issue I filed for an example.
He closed it without even addressing the issue of him closing the related issues that I explicitly brought up. There are scores of issues describing the same issue, but all of them are closed, and more of them are still being submitted. This has been going on since April of last year.
It's making me realize that sometimes a fork really is the only way to get around the problem that you're not in control of the project and there are bugs that many people need to be fixed.
Closing duplicated issues looks okay to me. The author not interested in fixing this issue himself looks okay to me. Is he refusing fix proposals about it?
To nitpick: It's not scraping, the youtube.com undocumented api changes all the time, in my tinfoil view probably to mess with things like youtube-dl and 3rd party apps.
yes - they don't generally access the underlying meta & media directly; they use the well established, stable and YT-prefered method of embeding a video. This doesn't work if you want direct file access to a specific resolution, or format, or audio only for example.
these platforms don't support youtube via a programmatic interface like this (and other libraries). They basically embed videos so the "API" is an http request for the traget url.
YT constantly changes how it masks access to the underlying resources with JS updates that break these libraries. The most frequent updates are to fix this.
The job of a maintainer is to reject PRs. So yeah, they are falling behind in saying no, maybe they just don't want to say no, so they let them sit. Happens everywhere.
Worth mentioning that youtubedl works with a lot of sites, not just YouTube. Most of those PRs are for other supported websites or requesting support for new sites.
Getting a fix merged to youtube-dl is quite an endeavour. No criticism to the fantastic maintainers - I encourage them to run their excellent project however suits them best - but I’m not going to try again in future.
Are PR visible to other users where they could make the same change to their local version even if the maintainer does not accept the PR in a timely manner? I know it is no where near the same, but just wondering.
Yes, on any PR you can click the source branch and see it. This is useful if a project is straightforward to build. You can just git clone the PR branch and build with that fix.
This seems indeed a confusing post/fork. youtube-dl is already very successful and popular. I use it almost every day.
So what is it about youtube-dlc that makes it special compared to the other 12k forks? You scroll down to the readme and it's the exact same, just that someone replaced -dl with -dlc.
I built a web app that is basically a wrapper around youtube-dl. By default it downloads and audio version that I can then just open with VLC on my Android phone and listen to in the background.
It's incredible how many links I can just throw at it, and things just work. Every once in a while I need to update the binary, but I can't blame that on youtube-dl.
YouTube-dl as a name is already confusing enough because it works for so many sites and YouTube is just one of them. Why complicate it even further in your fork by renaming it to YouTube-dlc (The "c" stands for community I guess) when dlc already stands for so many other things.
I assume it's going to be hard enough for the community to know about this fork in the first place, signaling direct continuity is definitely a reason to name the fork so similarly.
Besides, Mozilla has nothing to do with Mosaic, XBMC had nothing to do with Xbox for years...
Seconded. "youtube-dlc" might give an initial brand recognition boost from being similar to "youtube-dl", but it's the community that will make it or break it. Giving it a bad name (as argued by parent comment, and I agree) is not a good start.
I agree. Tho my main intention was backwards compatibility so that changes can easily be merged back into main project. There are also lots of people using it in their python projects. The naming discussion comes up every now and then as an issue.
So for now I would like the name youtube-dlc to stay. But depending on how things go in the future I may change the name.
I fully support this. I know it's hard for maintainers to do stuff for free and people request too much at times. But it appears that the attitude of the original maintainers of this project are actively "hostile" and they're not willing to accept the help from additional maintainers that have contributed fixes and testing. So a fork was in order, for sure. And from the look of the comments on the issues on the fork, it seems that people are happy with the continuity and effort that is being put into the fork.
Open source can be downright hostile and unpleasant at times, so I can definitely understand the frustration. All sorts of shenanigans happen constantly.
Yes it's stressing and time demanding. But you are absolutely right about how the maintainers treat other people.
At least it is open source. There are quite some closed-source projects that ran into troubles and basically disappeared.
I am not sure how they will react since the fork gained some attention now. However it's not my intention to let them look bad in any kind or act like I am better. All I want is a working solution where contributions by the community are acknowledged. But my attempt to openly point that out (issue youtube-dl dying?) basically got ignored and closed.
Fantastic. This is the beauty of open source. `youtube-dl` has had quite a number of breaking issues, where fully working PRs were raised and tested but not merged for weeks and with no communication.
I am glad to see `youtube-dlc`. Please, support the self updater as well (if it is not supported already). For my archival servers, I have a cron job that updates every day, I hope I can do that with youtube-dlc too.
Yes that is exact one of the reasons why I created this fork. The self updater should hopefully work in a future release. To be honest I haven't dealt with that yet. But feel free to open an issue so it's on my TODO-list.
It should be noted though that it has a fairly powerful built-in scrapper supporting video download from numerous sites. Unfortunately checking the code it seems to be closely integrated to the rest of the program, so someone cannot make (easily) a console utility based on it.
It's dodgy software. It got acquired and got some spyware/adware/toolbars embedded. Don't trust.
Personally, I removed the older version I had, fearing it would auto upgrade and auto install malware.
It's a download manager for shared hosting sites, mostly used for downloading movies/shows/games. It can get videos from youtube, dailymotion and a hundred sites but it's not the main purpose. Users are willing to try anything to get their latest tv show, there is too much money in bundling adware to ignore. The software was great once upon a time, then it got popular, eventually the author sold out.
Get the .jar, use only the .jar. It self-updates and everything. It's a shame about the crapware-laden installer, though, it's an excellent utility otherwise.
Seems like this is derived from a dissatisfaction with maintenance, but all I'm seeing is that they're maintaining things just fine. Yes, there are a lot of open PRs, but there are also quite a few merged, reviewed, and closed PRs. While I do support forks in the spirit of FOSS, I disagree that YT-dl is being poorly maintained and find that it continues to work well for my needs.
To the person who started the fork, what happened when you inquired about the apparent problem via email?
I wrote an email to all three core maintainers (one of them even twice) and to some of the inactive maintainers. There was no reply at all. However one reacted on github where I addressed the maintaining issues but basically just ignored it and all those who had similar concerns.
Well I am not the only one being frustrated about how things go. Especially the lack of communication. It's basically non-existent. At least the PRs that are merged are mostly either adult related or youtube. If your site is not even known to the maintainer - good luck. That will never be merged tho it may be one of the biggest sites in asia or so.
One tool that does everything. Its literally backwards. It eventually always results in one tool that's constantly broken.
There should be a tool for each site if there is a maintainer who actually cares about that site.
This is misapplying the unix philosophy. There would be too much duplicated functionality between different tools for different sites. It would be like having different versions of /bin/ls for different file systems.
To follow on, it's a valid point though. /bin/ls can handle different file systems because many other separate people/projects are responsible for the file system specific support (in the kernel).
Youtube-dl should be architected to allow many other people to own the site-specific support (i.e. via a plugin system, with the main "distro" of yt-dl including some popular plugins, which other people or themselves are responsible for), if it was like that then probably this fork wouldn't be necessary.
Yes that is a good point. The architecture is not fully suited to the current situation. There is actually already a base structure so it would only need the plugin system which pulls the missing extractors/downloader on the fly. Tho I may see problems here when it comes to windows binaries.
Another advantage would be to easier maintain those extractors as well as all issues that arise around specific ones. So no more thousands of issues addressing different extractors in one project.
"too much duplicated functionality"
What should that be? Who decides when its too much anyway?
No one cares about duplicated functionality in stuff they don't use. Probably 90% only want to download form YouTube. So why would they care if part of the code is used in another downloaders they don't use.
If you create a kernel/OS/Distro sure you don't want duplicated functions but this is just a downloader tool.
Also if there are generic functions that can be used by different tools you create a lib.
No one has to reinvent the wheel. Would it cause more work if summed up? Yes most likely but the work is shared and independent.
there are actually quite a few that uses youtube-dl. So far I don't know if anyone uses youtube-dlc. But I already got two messages that they want to build a webapp/service using youtube-dlc.
I'm having issues with youtube-dl and udemy (althought the Internet says it's possible. If anyone has some tip, I'll be very grateful). Does this one work with udemy?
The main issue I've found with youtube-dl is that "best" quality doesn't actually download the best video + audio in all cases, and might take a manual override. Not sure if youtube-dlc fixes this.
Nope. Maybe there is someone with a fix which can be merged. So far most people filter formats to their needs but that's not an option for basic users I guess..
108 comments
[ 3.8 ms ] story [ 163 ms ] threadA tiny conversation about this here as well https://github.com/blackjack4494/youtube-dlc/issues/56#issue...
Mention. it. in. the. code.
Adding a comment here and on Reddit is useless if nobody else sees it.
Explaining why a fork exists should be the first thing people read, especially how “everything is a fork” on GitHub.
Anecdote: youtube-dl still works for all the sites I'm using it for.
I'm not payed for this, I'm not even thanked for this most of the time, so I'm certainly not going to set any deadline.
But forking might be a solution, if you can give the necessary horse power for the long run.
Most can't.
The other day, I was trying to use a Python library to scrape some stuff. It kept breaking. I looked on the issues page and there were many, essentially useless bug reports asking them to fix the issue. I opened up the script and changed about 7 lines and it worked fine.
At times I imagine there could be some kind of "I know how to report bugs"-certification that people could somehow earn. Projects could then use this as a barrier to prevent getting too many useless issue reports.
The problem really goes both ways - as a software dev I'm disincentivized to report detailed bugs to projects, because there's a high risk it will get drowned out in the noise.
Some projects have quite specific templates for reporting issues. This helps a bit, but you still need someone to go in and remove issues that don't match the template, and few people want to volunteer for that work.
I mostly use GitHub to find off the shelf solutions to whatever problem I’m having or to avoid doing the work myself. If I run into any problems I have with something from there, I either try and fix it by tweaking the repository, using another, or just building the thing myself. I think the last time I reported an issue on an external repository was because of something I was made to use at work.
No code, no PRs, just a way for the public to create Issues.
Not sure what this means for the "culture of code" other than things are blurring...
https://github.com/ytdl-org/youtube-dl/issues/26462#issue-68...
Unfortunately, can't find a way to link to the edit in question directly. You'll have to click once there to see the diff.
In practice, I find it really useful to tidy up the top message of a bug report (but /really/ like seeing history of it). I also see cases where users spam raw logs into updates and tidying it up (perhaps making the log portion an attachment or changing formatting) makes it much clearer for future-you and other collaborators.
Are those issues you describe fixed in not merged PRs in youtube-dl and fixed in merged PR in youtube-dlc?
[1] https://github.com/ytdl-org/youtube-dl/pull/18230
[2] https://github.com/ytdl-org/youtube-dl/pull/23824
[3] https://github.com/blackjack4494/youtube-dlc/pull/55
Not saying forks should never happen - streamlink and KeepassXC are examples of great forks off the top of my mind - but you can't judge just on anecdotal stale PRs.
ydl has 700 open PRs, not 25k+. ydl hasn't had 25k+ PRs in its lifetime either, it's had 4k: GH issues and PRs are the same object, so the sequence is shared.
It's not like the forker inherited all the open issues/PRs, they just dropped them all.
Getting popular and having the masses leave a bunch of issues/PRs is hard. Reviewing code is hard and takes more expertise than writing it. A lot harder than reviewing your own PRs on a motivated (for now) fork.
That's wrong, most of the recent commits in the fork are merging branches from a lot of other repos. GitHub doesn't have a mechanism for inheriting a project's pull requests when you fork, but that doesn't mean they're automatically "dropped", since you can still merge the corresponding branch.
https://github.com/blackjack4494/youtube-dlc/commits/master
Although totally agree reviewing can be such a pain..
As someone who has tried to contribute to YouTube-dl in the past, the absolute disinterest I received and complete lack of follow up on simple bug-fixes caused me to completely dismiss YouTube-dl as a FOSS project worthy of my time.
And I say that as someone who sends PRs to projects I’ve visited once, over tiny things like typos or lack of installation/build instructions in the readme.
Some of my PRs received a crazy all-or-nothing treatment, asking me to fix huge, long-standing bugs nobody had fixed for years, instead of accepting a 2-line diff which did a provider-specific workaround which solved a real bug, but seemingly not in the “right” way.
Completely toxic and hostile.
That someone has started a community-fork with a more welcoming and pragmatic approach does not surprise me in the least.
Maybe I'll have more luck with this fork.
https://github.com/ytdl-org/youtube-dl/issues/23860
He closed it without even addressing the issue of him closing the related issues that I explicitly brought up. There are scores of issues describing the same issue, but all of them are closed, and more of them are still being submitted. This has been going on since April of last year.
It's making me realize that sometimes a fork really is the only way to get around the problem that you're not in control of the project and there are bugs that many people need to be fixed.
Verbatim from my comment: "the youtube.com undocumented api"
Thanks for reading.
In fact I'm pretty sure I've seen youtube-dl mention downloading the HTML page when using it with YouTube, so maybe it does even scrape YouTube.
Please explain.
YT constantly changes how it masks access to the underlying resources with JS updates that break these libraries. The most frequent updates are to fix this.
Keeping software well maintained involves a lot of saying no, unfortunately.
Right... and the youtube-dl maintainers aren't doing that.
So what is it about youtube-dlc that makes it special compared to the other 12k forks? You scroll down to the readme and it's the exact same, just that someone replaced -dl with -dlc.
It's incredible how many links I can just throw at it, and things just work. Every once in a while I need to update the binary, but I can't blame that on youtube-dl.
Besides, Mozilla has nothing to do with Mosaic, XBMC had nothing to do with Xbox for years...
Open source can be downright hostile and unpleasant at times, so I can definitely understand the frustration. All sorts of shenanigans happen constantly.
I am glad to see `youtube-dlc`. Please, support the self updater as well (if it is not supported already). For my archival servers, I have a cron job that updates every day, I hope I can do that with youtube-dlc too.
Personally, I removed the older version I had, fearing it would auto upgrade and auto install malware.
It's a download manager for shared hosting sites, mostly used for downloading movies/shows/games. It can get videos from youtube, dailymotion and a hundred sites but it's not the main purpose. Users are willing to try anything to get their latest tv show, there is too much money in bundling adware to ignore. The software was great once upon a time, then it got popular, eventually the author sold out.
https://jdownloader.org/jdownloader2
It takes a second to get how it works, but you can update youtube-dl from within the gui and it handles queues and all sorts of things really well.
https://github.com/mpv-player/mpv/blob/master/player/lua/ytd...
To the person who started the fork, what happened when you inquired about the apparent problem via email?
Well I am not the only one being frustrated about how things go. Especially the lack of communication. It's basically non-existent. At least the PRs that are merged are mostly either adult related or youtube. If your site is not even known to the maintainer - good luck. That will never be merged tho it may be one of the biggest sites in asia or so.
Youtube-dl should be architected to allow many other people to own the site-specific support (i.e. via a plugin system, with the main "distro" of yt-dl including some popular plugins, which other people or themselves are responsible for), if it was like that then probably this fork wouldn't be necessary.
Another advantage would be to easier maintain those extractors as well as all issues that arise around specific ones. So no more thousands of issues addressing different extractors in one project.
[1] https://github.com/FaisalUmair/udemy-downloader-gui