You mention you're limited to short videos due to issues with the maximum size of images on ios. Why not just generate multiple images? That way you could load the video "parts" in parallel as well.
On iOS Safari you're limited to 2 parallel HTTP connections at a time so not a huge advantage to the multiple connections. Don't disagree with splitting it up though
Decoding an MPEG1 video is of course much more taxing on the CPU than scrolling through a large image, but low-res videos shouldn't be a problem even on older mobile devices. jsmpeg uses WebGL to speed up the decoding process a bit. On an iPhone5S, it can decode 720p video in realtime[1].
jsmpeg only unpacks the current frame into memory instead of decoding the whole video at once, so it's memory footprint should be much lower. MPEG1 files are also much smaller than storing individual JPEG frames.
because of how easy it would be for the already bloated ad laden mobile web to abuse those powers, and how seriously obnoxious, bandwidth eating, and battery killing the results would be.
To ...abuse the powers of having access to an API for decoding video streams?
As we've already established, this can already be done, badly, with native javascript. So adding that access would add nothing new to "abuse" or "eat bandwidth" than what we already have, and will induce even LESS bloat than the current situation.
As for merely playing movies, ads etc (without direct access to low level decoding APIs), obviously javascript can already do that with the HTML5 media api + native encoders. So nothing new here either.
So I can't tell what disastrous "results" you're referring to stemming from what I propose.
Giving JS access to low level movie decoding/encoding would just be something to use to make creating JS-based video editors and such easier and more performant.
You can do it, but as you can see, it's not very efficient and eats battery. nobody would actually put this "decode video in javascript" stuff in an ad, as it stands. Making this stuff work well is hard.
make it efficient and easy, and you change the economics of the situation to encourage abuse.
>You can do it, but as you can see, it's not very efficient and eats battery. nobody would actually put this "decode video in javascript" stuff in an ad, as it stands
No, but they can already show any arbitrary video they want by using Javascript to add any number of videos playing on the website -- that's just plain vanilla HTML5.
Adding hooks to low level decoding doesn't add anything new that they can't already do when it comes to showing ads. For ads they only care for high level decoding.
It only adds something new when it comes to creating video mangling apps in JS.
that is on desktop. It is currently not possible to autoplay video on an iphone, and that is in fact the topic of this thread, and (partially) the purpose of this library.
Beyond autoplaying, I can't imagine what you mean about adding "hooks" to low level decoding. What does that actually gain you over the current situation?
>that is on desktop. It is currently not possible to autoplay video on an iphone, and that is in fact the topic of this thread, and (partially) the purpose of this library.
Aha, I see, thought that wasn't the case.
Now, having that capability didn't turn out to be much of a problem on the desktop, did it? I mean, regardless of battery concerns etc, do we really see that many autoplayed videos?
>What does that actually gain you over the current situation?
The ability to decode individual frames, re-arrange them, add filters to them etc using accelerated native APIs -- much more than some "play()", "skip()" affair.
well actually the ability is a huge and obnoxius problem on desktop, particularly for accessibility. there are lots of autoplaying videos. i do not know what web you are on where there isn't. I'd be happy if the deaktop browsers turned off autoplaying video and audio. or at least add that as an option for me, the user who has to jump for the mute button every time some jerk developer thinks they are being clever. aside from that, mobile is a different story- I doubly don't want a loud car salesman voice telling me about his get rich quick scheme on the train ride home- eating my precious mobile bandwidth, costing me real money.
as for your special api abilities, again, you can do all those things now, even on iphone. so again, what more do you actually want?
Could be interesting if you wanted to do client-side filtering/modifications, but a lot of that is probably possible with video files (and maybe WebGL) as well.
Apple's tech dedups 8x8 pixel blocks and creates a jpeg image that is a lookup table of 8x8 pixel blocks, using a json data object to figure out which blocks go where on each frame.
this thing just stores full frames in a giant jpeg and displays them.
if you wanted to replicate Apple's technique, there's a neat (but discontinued) app on the mac app store called phosphor that will take in movies or frames and output in the apple-like canvas/json/jpeg 8x8 pixel blob format.
31 comments
[ 0.72 ms ] story [ 72.8 ms ] threadThe world is in trouble
Ah what we tell dem murdera
Longer demo: http://phoboslab.org/files/jsmpeg/
jsmpeg only unpacks the current frame into memory instead of decoding the whole video at once, so it's memory footprint should be much lower. MPEG1 files are also much smaller than storing individual JPEG frames.
[1] http://phoboslab.org/files/jsmpeg/benchmark-720p.html
While killing the battery while doing it... Why we don't just expose lower level decoding apis to JS instead, I don't know...
As we've already established, this can already be done, badly, with native javascript. So adding that access would add nothing new to "abuse" or "eat bandwidth" than what we already have, and will induce even LESS bloat than the current situation.
As for merely playing movies, ads etc (without direct access to low level decoding APIs), obviously javascript can already do that with the HTML5 media api + native encoders. So nothing new here either.
So I can't tell what disastrous "results" you're referring to stemming from what I propose.
Giving JS access to low level movie decoding/encoding would just be something to use to make creating JS-based video editors and such easier and more performant.
make it efficient and easy, and you change the economics of the situation to encourage abuse.
No, but they can already show any arbitrary video they want by using Javascript to add any number of videos playing on the website -- that's just plain vanilla HTML5.
Adding hooks to low level decoding doesn't add anything new that they can't already do when it comes to showing ads. For ads they only care for high level decoding.
It only adds something new when it comes to creating video mangling apps in JS.
Beyond autoplaying, I can't imagine what you mean about adding "hooks" to low level decoding. What does that actually gain you over the current situation?
Aha, I see, thought that wasn't the case.
Now, having that capability didn't turn out to be much of a problem on the desktop, did it? I mean, regardless of battery concerns etc, do we really see that many autoplayed videos?
>What does that actually gain you over the current situation?
The ability to decode individual frames, re-arrange them, add filters to them etc using accelerated native APIs -- much more than some "play()", "skip()" affair.
as for your special api abilities, again, you can do all those things now, even on iphone. so again, what more do you actually want?
https://output.jsbin.com/qocefivowa
Are there other legitimate use cases than using this a fallback for iOS? Curious.
Previously: https://news.ycombinator.com/item?id=4531088
Teardown: https://docs.google.com/document/u/1/pub?id=1GWTMLjqQsQS45FW...
https://www.divergentmedia.com/phosphor
Whether you actually gain anything from using this technique as opposed to other simpler techniques like canvid, well, let's just say YMMV