> A real-time, node-based video effects compositor for the web built with HTML5, Javascript and WebGL
I for one am really impressed. While I'm not up to date with node.js, WebGL (or even HTML5 tbh), the boundaries that people keep pushing with web technologies is pretty wild.
EDIT: Reading more, I'm not sure if I am proud or embarrassed about my narrow focus over the last few years. I'm proud that I'm making a good living due to it, but disappointed that I haven't been able play with these things. Reading about WebGL [1], I feel like there is a new world to explore (for web programming).
I'm working on porting this over to Seriously.js so it should run in the GPU, hopefully pretty fast. The first crack at it was not so good, and now I'm experimenting with some new features that are only in Chrome nightly builds at the moment (and buggy at that). So look for this in the next 3-6 months.
I've clocked many hours into After Effects, but seeing this in a browser and _interactive_ is amazing. Never would I have thought that JS and browsers would progress so much in such a short period of time.
OTOH, it's mainly possible because computers are faster so Javascript isn't a slow language anymore, and the document model is just getting out of the way and letting you manipulate pixels directly.
This isn't manipulating the pixels directly. Javascript is still way too slow to do that, and I don't think that will ever change. The real work is done by the graphics card. The Javascript only uploads a frame to the graphics card, makes it run a shader and then displays the adapted frame.
"Sadly, we are unable to get Seriously.js to work on your computer. Sometimes WebGL gets a bit weird with certain graphics hardware and drivers. Please have a look here for more information."
And unfortunately the dynamically-compressed-to-shit song wasn't enough to provide entertainment on its own.
Running the latest Chrome on Mac OS X. (btw, realized my initial comment probably came off a little snarky. My apologies. From the sounds of the other comments, it's something incredibly cool!)
This is the sort of thing we're working on at Vidpresso. If this impresses you, and you want to build stuff like this... we should be friends. Contact info is in my profile.
No one said it would be MORE complicated than the examples.
"Would also be interesting" and "fun to compare" give no implication of relative difficulty. I would like to see the algorithm though so please post the code if you get it working in JS! That would be awesome!
It is actually slightly more complicated. The existing effects work in GLSL and are just transforming single frames of video. For that effect to work, you need the previous buffer states; this means that seeking will cause the image to be different, and you need to feed the previous render state back as a texture.
Well no, because each frame builds on all the frames before it. Either way, seeking is still broken unless you run through all the frames up to that point.
Each frame has the greenscreen turned into a fully transparent pixel, then gets blitted onto a buffer containing all of the existing buffer's data. That means that for each frame, you're building on the data from every frame that came before. This is fine if you're playing from beginning to end, but when you seek, you're going to break that continuity.
Yes, you do need to process from beginning to end. If you look at the original video, you see that each frame builds on the one before it, which builds on the one before that. Remember, the buffer is never cleared -- it's just overwritten with a few new pixels (the dude moving) each room.
You still need to remove the green-screen. This either means an expensive process on the CPU side, or pushing the frame to the GPU to remove it with a shader and then using RTT and rendering that or reading the framebuffer and drawing that to a canvas. Note, it's really not difficult to do this, it's just slightly more complex than the other effects, is all.
This is pretty awesome, but it seems like OK Go has cornered the market on "cool web tech" demo music videos. Doesn't anybody remember their collaboration with Google?
Do you mind sharing your video card info and operating system? It will be hard to debug remotely, but I'll see what I can dig up.
Please post it here or on github: https://github.com/brianchirls/Seriously.js/issues
Intel announced a year or so ago they could JIT compile javascript and run it on their GPU. Maybe that's why?! If so, welcome to the bold new age of GPGPU and AMD's HSA... :)
LOL, my old core2 duo has both cpus running over 80% on Ubuntu with a GeForce GTX260. But when I click on another tab, with the video still playing on a non-visible tab in Chrome, the cpus drop to around 20%. When I click on the video tab again, the cpu ramps up to the 80% mark again. Neat trick.
I'm running an i5 processor, and when the video was playing my cores jumped from the 5-10% range (background stuff I've got running) to the 25-40% range.
Interestingly, just switching from the demo tab to this HN tab (FF 21 on Ubuntu) drops the cores to the 10% range. They jump again when I switch back.
"Sadly, we are unable to get Seriously.js to work on your computer. Sometimes WebGL gets a bit weird with certain graphics hardware and drivers. Please have a look here for more information."
"Sadly, we are unable to get Seriously.js to work on your computer. Sometimes WebGL gets a bit weird with certain graphics hardware and drivers. Please have a look here for more information."
Yeah, same here, running Ubuntu on a Dell Precision laptop. Guess WebGL doesn't like my setup.
I coded WebGL a little over a year ago on a Precision "laptop" under Suse Linux. What I needed: (A) up-to-date Nvidia proprietary drivers (B) "de-blacklist myself" somewhere deep in the browser's hidden WebGL config page (I think it was about:gpu or some such back then in Chromium, but I'm not really up to date).
Yeah, I run Chrome on Ubuntu on a pretty powerful Lenovo T-something with an NVIDIA whatever optimus pain in the neck. The only way I can get it to work is to add --ignore-gpu-blacklist to the Chrome command line.
Check:
/usr/share/applications/google-chrome.desktop
(or something like that)
You should see some lines that start with "Exec" where you can add that parameter to the end.
I had the same deal with Chrome on my OSX Mountain Lion. Here's what fixed it for me:
1. Visiting chrome://flags
2. Scrolling down to Disable WebGL, hitting "Enable", and then "Relaunch now"
3. Repeating steps 1 and 2, but hitting "Disable" rather than "Enable".
Whoa, this caused the second BSOD I've seen since Windows 98. When I started messing with the options, the windows just flashed wildly for a while, some popup about ATI Radeon driver errors was struggling to spring to existence, and then everything froze.
This is awesome! I can't wait to see the mashups that spring forth from this (and hopefully make one or two myself). There will definitely be something really creative soon. Thanks for doing so much legwork on this.
Couldn't get it to work in FF (Mac/22.0) - it asked for camera permission, camera activity light went on, but no images and constant re-prompts for permission. But, worked quite well in Chrome (Mac/27.0.1453.93).
Nice work. I'm curious about where you got the video, was that something you recorded, or got from somewhere? I don't really work with video, but I'm sure I can find something useful for it in the future.
I kinda wish it was at a higher bit-rate. There are some jaggies at the edges of the hair and stuff that I had to cover up by tinting the color of the video.
195 comments
[ 3.2 ms ] story [ 239 ms ] threadI for one am really impressed. While I'm not up to date with node.js, WebGL (or even HTML5 tbh), the boundaries that people keep pushing with web technologies is pretty wild.
EDIT: Reading more, I'm not sure if I am proud or embarrassed about my narrow focus over the last few years. I'm proud that I'm making a good living due to it, but disappointed that I haven't been able play with these things. Reading about WebGL [1], I feel like there is a new world to explore (for web programming).
[1] http://en.wikipedia.org/wiki/Webgl
(discussion: https://news.ycombinator.com/item?id=5682831)
I'm working on porting this over to Seriously.js so it should run in the GPU, hopefully pretty fast. The first crack at it was not so good, and now I'm experimenting with some new features that are only in Chrome nightly builds at the moment (and buggy at that). So look for this in the next 3-6 months.
Maybe if you're already an after-effects wiz this doesn't seem as magical, but it certainly felt like magic to me.
They're famous for quirky video clips (original for this song is at http://youtu.be/12zJw9varYE). You should watch http://youtu.be/qybUFnY7Y8w . You'll like it. Trust me.
Incidentally, you really aught to see how they make music with a car... http://www.youtube.com/watch?v=MejbOFk7H6c
It was filmed on a green screen for a different effect: Frame layering.
And unfortunately the dynamically-compressed-to-shit song wasn't enough to provide entertainment on its own.
i assume since no one else has mentioned this that it's just me: http://i.imgur.com/TdQCpsR.jpg
If that's not the case, do you mind writing a bug issue on github with your browser details?
https://github.com/brianchirls/Seriously.js/issues
Thanks
https://aboutmybrowser.com/
Firefox 21.0
(Note: This comment was originally complaining that the link didn't work in Firefox; edited when I found the cause.)
http://okgo.net/2010/01/20/wtf-video-remix-project/
That doesn't seem more complicated than the examples they showed.
"Would also be interesting" and "fun to compare" give no implication of relative difficulty. I would like to see the algorithm though so please post the code if you get it working in JS! That would be awesome!
http://imgur.com/a/PQX63#1
It's full of lots of (mostly green) noise because the demo's chroma key algorithm/parameters aren't that good, but it was fun to do!
http://allisnotlo.st
I fully expect them to do something with Microsoft for IE11 to complete the cycle.
Actually there have been some other very nice ones recently:
http://donottouch.org/
http://lights.elliegoulding.com/
Edit: Here is what it looks like for me: http://i.imgur.com/8wflg8Q.jpg in both Firefox 21 and Chrome 29+27
Do you mind sharing your video card info and operating system? It will be hard to debug remotely, but I'll see what I can dig up. Please post it here or on github: https://github.com/brianchirls/Seriously.js/issues
Thanks
http://i.imgur.com/OF58RZl.png
After the peaks when it is flat at around 3% is when it is playing.
Core 2 Quad Q9950 and an nVidia Geforce 8800 GT 512MB.
As for 80%, umm... video driver issues? :-(
Interestingly, just switching from the demo tab to this HN tab (FF 21 on Ubuntu) drops the cores to the 10% range. They jump again when I switch back.
Yeah, same here, running Ubuntu on a Dell Precision laptop. Guess WebGL doesn't like my setup.
Check: /usr/share/applications/google-chrome.desktop (or something like that) You should see some lines that start with "Exec" where you can add that parameter to the end.
override software rendering list
1. Visiting chrome://flags 2. Scrolling down to Disable WebGL, hitting "Enable", and then "Relaunch now" 3. Repeating steps 1 and 2, but hitting "Disable" rather than "Enable".
Please keep the bug reports and questions coming. I'll try to fix/answer what I can.
Although, both Firefox and Chrome have been a bit buggy for me with getUserMedia.
Try the Ascii Text effect.
Couldn't get it to work in FF (Mac/22.0) - it asked for camera permission, camera activity light went on, but no images and constant re-prompts for permission. But, worked quite well in Chrome (Mac/27.0.1453.93).
Nice work. I'm curious about where you got the video, was that something you recorded, or got from somewhere? I don't really work with video, but I'm sure I can find something useful for it in the future.
I kinda wish it was at a higher bit-rate. There are some jaggies at the edges of the hair and stuff that I had to cover up by tinting the color of the video.