not every video frame is encoded in RGB. actually, very few transports use that.
the in-memory layout of a YUV-encoded frame is different than the one of a RGB one. it needs conversion, which implies an intermediate copy, as well as traversing a buffer that can go up to 1080 rows.
most video players (and Flash on Windows and MacOS), these days, use shaders and multi-texturing to do that conversion on the GPU; it removes a copy, because you just push the YUV frame as it is to the GPU - and your CPU can go back to a low C state.
I think Flash on Linux does this as well, on nVidia: the blog post linked is from 2010. the way Adobe detect capabilities is, in itself, hilariously bad - instead of checking the GL required GL extensions like any normal people, they do a check on the GL vendor string. the Mesa guys pointed that out, but I doubt they ever received a response.
Are you on Linux ?
That is the only plat that I see taking higher cpu usage while playing viedo in flash. Both windows and osx now play flash with little cpu usage.
I only skimmed the article, but that strikes me as a "mistranslation".
Adobe is emphasizing that they have to do more compositing logic, so they they need an RGB version of the video frame on the CPU side.
If they did the colorspace conversation on the GPU, they'd have to pull the converted image back and incur the latency hit. Apparently, they see less latency in doing the conversation on the CPU, and have made the call to trade processor efficiency for latency.
To be clear: YUV colorspace conversion on a GPU is really damn simple. I have a ~30 line shader that does it from my video processing codebase. But I can take the hit - I do a substantial amount of image manipulation using shaders on the GPU, and mask the read latency with heavy multi-threading. A Flash application doesn't have this luxury.
But this discussion is largely academic - if you're writing a Flash based video player that doesn't need the flexibility of Flash's full compliment of image manipulation and composting functionality, you'd be using their StageVideo API - an API that does do all the video work on the GPU. This API was introduced in Flash 10.2, which came out after this article from 2010.
the internet at large has unanimously declared that it appreciates the Flash Player’s solution to the video problem
Really? The Internet at large is able to unanimously declare things, and one of those things is that it appreciates Flash's solution to the video problem? Tell me more...
Besides that it's a shame that the option to just get the stream running in VLC is such a hassle (with VLC I get much lower CPU usage + the ability to freely move and resize the video and place it somewhere on my second screen, even fullscreen)
Wow. A tip to anyone reading this: if you ever find yourself explaining away poor—nay, pathological—performance with anything other than ‘we made some mistakes in the past and we’re doing what we can to correct them’, please take a good hard look at yourself and consider exiting the software industry. That’s all I have to say.
Edit: that wasn’t all I had to say.
> What about Mac? I’m not sure but my Mac colleagues have mentioned something about Apple not making their hardware decoding APIs available to applications
Well why didn’t you just go ahead and ask them before writing a blog post on company letterhead?! Is the corporate structure here so screwed up that you can’t actually do that, or are you just lazy?
Doubly so if your product crashes even half as often as Flash. That they cannot seem to code for stability doesn't give me any confidence in their ability to code for speed.
I heard that they refused to work with Apple to improve its performance on Mac, and that was a core reason why they got excluded from the iOS platform -- searching for this I came up with an explanation directly from Apple:
"Symantec recently highlighted Flash for having one of the worst security records in 2009. We also know first hand that Flash is the number one reason Macs crash. We have been working with Adobe to fix these problems, but they have persisted for several years now. We don’t want to reduce the reliability and security of our iPhones, iPods and iPads by adding Flash."
If that's the major reason preventing them from being performant, then I misunderstood the reason for crashing and the reason for poor performance to be the same. Apple should not be complaining about CPU usage.
That doesn't excuse Adobe from making buggy plugins.
Indeed the flash plugin has been riddled with crashers and security vulnerabilities. But the blame for high CPU usage during video decoding seems to have been mostly in Apple's court before that.
With Mountain Lion 10.8 Flash is still the same like before those APIs were released. I think Adobe just don't cares about it. I really I can't the strategy behind it. Flash at its best times was nearly on every PC and Mac why is that they didn't want to improve the quality of their app.
I don't want to listen (to customers and developers) you need to feel it.
That quote is a great piece of PR. The number one reason quoted from that Symantec report was: "Microsoft Windows SMB Remote Code Execution". It also didn't have anything to do with what makes Macs crash, it was a report on vulnerability per plat, it mentioned flash in cross site exploits, not flash by itself. But to this day and for a long time the music keeps playing.
> Flash Player solves a different problem: It plays linear media files from start to finish while combining the video with a wide array of graphical and interactive elements (buttons, bitmaps, vector graphics, filters)
And therein lies the problem with Flash video: dozens of video player implementations, each frustratingly different and user unfriendly than the next. People are building custom chrome on top of HTML5 video too, unfortunately, but at least it's standardized (and semantic) which allows users to consume web video with any interface they want.
I don't know if the explanation is accurate or just their own product defense, however (long) after playing a video (and not something else in flash), I usually go into top and kill the Chrome flash process manually as it's a) taking an insane amount of memory b) taking a lot of CPU for no reason (I'm not watching the video anymore and closed the page). Seems there is a ton of leakage going on after playing a video. So even though the story might to explain the actual playing of the video, it doesn't explain the molestation of my i7/8gb/ssd machine AFTER I closed (the only) Flash window. Not to mention the crashes.
The message that I take is that some companies use Flash as the wrong tool to solve a problem which I'm not interested in. I want to see videos with minimum overhead. I don't want to be bothered with ads, poor performance, poor user interfaces, poor security, ... all which I get through Flash.
Given the options available to them (roughly: expensive real video platform, expensive quicktime video platform, cheap content creation platform (in the form of flash) that can do video), those companies (and let's be honest, there's really only one that matters, youtube) made a perfectly reasonable decision. You might moan about flash interfaces, but unless you're willing to pay more (or, equivalently, view more ads) for a real, performant video interface (which was a genuine option for a while - e.g. the now-defunct stage6), you can't blame them for making that choice.
(I feel it's a real failure of vision in the open-source community that there wasn't a free theora (say) plugin available for all browsers, and server-side video hosting platform, when youtube was getting started.)
h264 has been a standard for a long time and performs very well on current hardware (many machines have hardware acceleration for it). Google has wasted terawatts of electricity due to flash.
Again, if users cared enough they would vote with their feet. It seems like no-hassle, low-ads video playback is worth more to us than playing it more efficiently.
Not really, StageVideo has a bunch of limitations in the compositing pipeline, but that doesn't really matter if all you want to do is build a 2D movie player.
Anyways, I guess Adobe changed their mind later, and decided they wanted to solve both of the "different problems" mentioned in the blog post.
(Remember, Flash used to be all about vector animations. Sometimes it'd seem like the ubiquitous flash-as-a-video-player thing happened almost by accident; it turns out the video support (a relatively small feature in the whole flash featureset) was good enough to bring "universal" video to the web.)
tl;dr - we accidentally a whole software video decoder to show some circles and rectangles at the bottom. and so that you can haz internet when playing video.
Not to mention that they were flat out lying when they claimed other players don't do this. Anyone who remembers when Mac devs were hand-tuning AltiVec implementations of YUV-RGB routines knows that Adobe was just choosing not to spend the money.
Yeah, that's pretty dubious. I was under the impression that all video players do that anyway - that video is encoded in the YUV color space and always converted to RGB on playback.
To blame that for their poor performance suggests that their YUV-RGB conversion code is among the slowest in the industry.
If only there were a way to let a plugin give VLC a URL to open and overlay over a specific part of your screen until you close the tab.
Nah, that's crazy talk.
Who would want to watch a high-quality hardware-accelerated local video that gets deleted when you close the tab, from the familiar interface of a web browser? Crazy talk, crazy talk.
Don't forget that Flash also solves the problem of denying access to your selected media until unskippable ads have played, or overlaying said ads on top of the media you're trying to watch.
"Try to keep up here."
"Again, say it with me:"
"Please tell me you know people who aren’t as tech savvy as you. If you don’t, get to know some in order to maintain proper perspective."
A lesson in how to not communicate a technical idea.
This comment on the blog itself perfectly sums up my feeling about the article and its content, so I'll quote it here:
"Again, say it with me…
I’ve been reading this blog since the beginning. I always thought you were a little condescending, but you seemed knowledgeable enough and I thought you were doing good work.However, you can say this with me: Eat shit."
49 comments
[ 2.3 ms ] story [ 117 ms ] threadThe screen has red, green and blue pixels, either way a video is played, the colours have to be converted into RGB.
the in-memory layout of a YUV-encoded frame is different than the one of a RGB one. it needs conversion, which implies an intermediate copy, as well as traversing a buffer that can go up to 1080 rows.
most video players (and Flash on Windows and MacOS), these days, use shaders and multi-texturing to do that conversion on the GPU; it removes a copy, because you just push the YUV frame as it is to the GPU - and your CPU can go back to a low C state.
I think Flash on Linux does this as well, on nVidia: the blog post linked is from 2010. the way Adobe detect capabilities is, in itself, hilariously bad - instead of checking the GL required GL extensions like any normal people, they do a check on the GL vendor string. the Mesa guys pointed that out, but I doubt they ever received a response.
There is no excuse for that - I could understand higher CPU usage, but not by factor of 3.
Adobe is emphasizing that they have to do more compositing logic, so they they need an RGB version of the video frame on the CPU side.
If they did the colorspace conversation on the GPU, they'd have to pull the converted image back and incur the latency hit. Apparently, they see less latency in doing the conversation on the CPU, and have made the call to trade processor efficiency for latency.
To be clear: YUV colorspace conversion on a GPU is really damn simple. I have a ~30 line shader that does it from my video processing codebase. But I can take the hit - I do a substantial amount of image manipulation using shaders on the GPU, and mask the read latency with heavy multi-threading. A Flash application doesn't have this luxury.
But this discussion is largely academic - if you're writing a Flash based video player that doesn't need the flexibility of Flash's full compliment of image manipulation and composting functionality, you'd be using their StageVideo API - an API that does do all the video work on the GPU. This API was introduced in Flash 10.2, which came out after this article from 2010.
Really? The Internet at large is able to unanimously declare things, and one of those things is that it appreciates Flash's solution to the video problem? Tell me more...
Might be due to the implementation of the sites (german broadcasters ard & zdf) but flash works pretty awesome for me.
Edit: that wasn’t all I had to say.
> What about Mac? I’m not sure but my Mac colleagues have mentioned something about Apple not making their hardware decoding APIs available to applications
Well why didn’t you just go ahead and ask them before writing a blog post on company letterhead?! Is the corporate structure here so screwed up that you can’t actually do that, or are you just lazy?
I heard that they refused to work with Apple to improve its performance on Mac, and that was a core reason why they got excluded from the iOS platform -- searching for this I came up with an explanation directly from Apple:
"Symantec recently highlighted Flash for having one of the worst security records in 2009. We also know first hand that Flash is the number one reason Macs crash. We have been working with Adobe to fix these problems, but they have persisted for several years now. We don’t want to reduce the reliability and security of our iPhones, iPods and iPads by adding Flash."
Apple did not open their video acceleration APIs until they released OS X 10.6.3 circla mid-2010.
http://xbmc.org/davilla/2010/05/03/osx-gets-h-264-accellerat...
That doesn't excuse Adobe from making buggy plugins.
I don't want to listen (to customers and developers) you need to feel it.
Worse yet, nothing stops flash from working in the YUV color space when mixing video and non video elements.
Much more importantly, it doesn't give me any confidence in their ability to code for security.
Here is a good analysis of that text that explains a lot on how that statement is great PR but nothing more: http://truegryc.blogspot.pt/2010/05/response-to-thoughts-on-...
This is a two and a half year old blog post. Mac hardware video decoding is supported now.
And therein lies the problem with Flash video: dozens of video player implementations, each frustratingly different and user unfriendly than the next. People are building custom chrome on top of HTML5 video too, unfortunately, but at least it's standardized (and semantic) which allows users to consume web video with any interface they want.
(I feel it's a real failure of vision in the open-source community that there wasn't a free theora (say) plugin available for all browsers, and server-side video hosting platform, when youtube was getting started.)
http://www.adobe.com/devnet/flashplayer/stagevideo.html
Anyways, I guess Adobe changed their mind later, and decided they wanted to solve both of the "different problems" mentioned in the blog post.
(Remember, Flash used to be all about vector animations. Sometimes it'd seem like the ubiquitous flash-as-a-video-player thing happened almost by accident; it turns out the video support (a relatively small feature in the whole flash featureset) was good enough to bring "universal" video to the web.)
To blame that for their poor performance suggests that their YUV-RGB conversion code is among the slowest in the industry.
Nah, that's crazy talk.
Who would want to watch a high-quality hardware-accelerated local video that gets deleted when you close the tab, from the familiar interface of a web browser? Crazy talk, crazy talk.
Then again, I don't use it because I couldn't get any seek UI to appear.
That problem is largely tracking user behavior across the internet.
"Try to keep up here." "Again, say it with me:" "Please tell me you know people who aren’t as tech savvy as you. If you don’t, get to know some in order to maintain proper perspective."
A lesson in how to not communicate a technical idea.
"Again, say it with me…
I’ve been reading this blog since the beginning. I always thought you were a little condescending, but you seemed knowledgeable enough and I thought you were doing good work.However, you can say this with me: Eat shit."