If I understand it correctly a lot of the geometry effects build on the ability to do texture reads in the vertex shader (OpenGL calls it "vertex texture fetch"), a not much-noticed, but incredibly powerful feature of modern WebGL implementations.
The reason it is so powerful is because one texture can be used as a write target for the fragment shader and as a read target for the vertex shader, essentially creating a feedback loop that lives entirely on the GPU.
Not all browsers support the feature though (check the MAX_VERTEX_TEXTURE_IMAGE_UNITS constant). Mobile devices could be problematic too since most (if not all) OpenGL ES 2.0-era devices don't support it in hardware.
Still this is one of the most impressive WebGL demos I've seen. Fantastic stuff.
The major weakness in using textures as intermediate targets is the loss of precision from the texture formats as well as the intermediate values. OpenGL ES 2 (and thus WebGL) does not require a full 32 bit floating point pipeline, so the results may vary if you run on mobile devices (that are not the latest generation GL ES 3.x devices).
In proper OpenGL, you'd be able to use transform feedback to write in to buffers with no loss of precision. And using buffers is less limited than texture fetches in the vertex pipeline.
For applications where precision matters (ie. everything scientific), WebGL on GLES2 devices is a no-go. WebGL standardization should pick up the pace to better match the development of OpenGL.
It's a bit of a shame that WebGL settled for the lowest common denominator (i.e. OpenGL ES 2.0 capabilities).
This was probably to enable WebGL on mobile devices that would have otherwise been locked out, but it heavily restricted things on the desktop which for the most part would have OpenGL 4 capable GPUs these days.
However given that WebGL on mobile still mostly sucks anyway, not sure if going for the lowest common denominator was the right decision.
WebGL 1.0 is almost 4 years old. OpenGL ES 2.0 was then latest and greatest.
And WebGL 1 has taken this long to reach mostly-working in implementations, it would have probably died in the crib if it had targeted the nascent GLES 3 feature set.
Running GLES shaders safely and reasonably fast in a sandbox (on top of insecure & crash prone drivers) is high wizadry.
Laptop or mobile? How much time did you spend watching the examples? What is your battery's storage capacity (milliamp-hours, usage hours, etc.)? Does it have a GPU? CPU usage? Is that battery drain consistent w/ other things that use that amount of CPU? etc.
Totally agree, under Chrome / OS X, fans went crazy on my rMBP 15 iGPU and CPU raised 80 C degrees just to watch a few slides of that site.
I closed the tab to stop overheating.
Huh. I use Safari / OS X on a 4 year old MacBook Pro. I keep it locked on the integrated CPU and it worked fine, barely increasing the system load. Only one or two of the effects near the end made any noticeable difference.
Of course you're pushing 4x as many pixels as me.
Try Safari (you have to enable WebGL in the Develop menu). I wonder it's a Chrome issue.
This is one of the most beautiful things I've seen for some time. And to think this is all in a browser, usable from JavaScript. I feel like there could be so many applications for this, for more complex, interdependent visualizations, yet easier than D3 and the like. Also, in the end it's described as Reactive DOM. So, now I wan't to see TodoMVC redone with this. It must be the fastest yet (I'm only half joking!).
I wonder what it needs to handle text presentation and input. HTML overlays are mentioned. Perhaps there are already WebGL text renderers that could be integrated. Of course visualizations this complex make my Macbook scream, but that's all right since I'm seeing something new (in a browser) and delightful. I have a few million data points that could benefit from vantage point like this, which need complex dependencies and controls.
To handle HTML overlays, I basically need to add read back capabilities to find the final on-screen positions of points, so I can sync with CSS 3D matrices. GL text is a rabbit hole I'd prefer to avoid, especially since I often need math notation. It would turn into HTML/CSS-for-GL right away.
Right now, very dead. It was more of an idea than a real thing, badly architected, but with some good ideas waiting to be reimplemented on a non 0.x stack. Still perpetually disappointed every new "neo terminal" is monospace tho.
> Still perpetually disappointed every new "neo terminal" is monospace tho.
Have you thought of ways around the path dependence[1] on monospace imposed by existing bodies of textmode UIs (and source code)? It seems unlikely that a new terminal-esque tool would succeed without some kind of legacy support. The best concept I've come up so far with is to build in affordances which handle legacy vs. new-world user interaction and app I/O models.
Related, I continue to hold out (vain) hope that elastic tabstops[2] will someday gain traction.
Formatting legacy stuff was always part of the deal, but at the same time, I was never interested in being able to host vim. Some people disagreed rather vocally.
One of the things I discovered was just how much legacy cruft is really around. Not just things like ANSI colors, but e.g. grotty syntax. It made no sense until I realized it was created for teletype printers... it underlines things by backspacing after every character and printing a "_". It bolds by backspacing and repeating the character. I had to parse this to support man pages, and I assume the default TTY still does too.
The other thing was that so much of Unix workflow really only works by accident. The fact that you can ssh + sudo + ssh + ... is because the pipes are too dumb to fuck it up. Take for example SSH escape sequences... [1] they only work on the first hop. The proper solution is out-of-band signaling.
From an architecture point of view, the whole termcaps / stdio thing is crazy. The Unix principle is supposed to be about simple agnostic composition, and yet most tools have to sniff out their environment in order to maintain this illusion. Text files are for people, not machines. And if you want to see a never ending discussion, just ask a bunch of greybeards how to write a shell script that can handle files with spaces in their name.
Sort of unrelated, but when I saw TermKit I couldn't help the surface similarities with PowerShell. In fact, I believe what you had there could almost work as a PowerShell host, although things like http://poshconsole.codeplex.com/ share some of the same ideas.
I'm pretty excited about it. I think there are three impressive things about it.
First is that you can write vertex shaders in a reactive DOM. That makes it much easier to get pictures up on the screen. If any of you have ever messed around with vertex shaders, it can be a bit of a nuisance.
Second is that while the reactive DOM doesn't really exist as XML, it can be expressed as such, and would be easily diffable. This is important for collaboration.
Lastly, because it's making the GPU do all the work, data visualizations can be done by pushing large amounts of data to it. We should be able to see more patterns from data as a result.
The visual representation of calculus, speed, velocity, and acceleration taught me more in 60 seconds than would take in 4 hours worth of lectures. Fantastic! (Makes my laptop catch on fire though)
Looks great, seems like this takes advantage of implicit calculations a lot. For example, there are two ways to draw a graph:
Calculate just the points to be drawn, then draw them (explicit generation).
Calculate the entire surface/volume, and draw values where they exist (or based on magnitude or whatever properties are used) (implicit generation).
The second method is in some circumstances less efficient, especially if the graph is very simple and takes up little screen space, but overall much easier to work with. Its similar to the difference between ray casting and rasterization, in a way.
Yes and no, everything is still sampled on grids. But the intermediate calculations can be doing tons of implicit look ups. So it's more like lazy evaluation, though there's no auto-memoizing (because the memory/time tradeoff is highly context dependent).
So if you wanted to render an implicit surface this way, you could do e.g. marching cubes or tetrahedra on a grid, and only feed in a scalar 3D field, either as an array or as a procedural function. Or you could do a <raymarch> operator for raymarching a distance field. On the inside, this could be a dumb per-pixel loop, or do recursive quad-tree subdivision. You shouldn't need to care.
It's all vaporware right now, but it's just a matter of fitting it in neatly.
His website [1] is one of the most impressive websites on the internet. Famo.us got nothin' on him!
First time I heard about Steven was when I saw this [2] post last year.. the best part is that he leaves many easter eggs or "achievements" around for you to discover :)
While we (at famo.us) have had our fair share of neckbeard faux pas, we and Steven are both fighting for the same future of the web; one where more sane low level primitives, such as a proper scene graph, are exposed to developers as a foundation upon which we can build better libraries and frameworks like MathBox 2. [0][1]
These is no need to make this into a pissing contest or a rivalry. We're fans of Steven's work and incredibly impressed with how he has pushed the state of the art on the web forward. Anyone who works on the bleeding edge like this helps build a brighter future for the web and creates more knowledge upon which others may build. Anyways, please keep the discussion focused on what Steven has achieved here instead of trolling.
Any plans for Oculus support? I'm building a code analysis framework with a visualization tool and if MathBox were to support the Rift it would be a no-brainer over using raw SVG or D3.
MathBox 2 is built on top of threestrap (don't google it, you get shoes), to enable exactly this kind of extensibility without me having to do it all myself. Just by following a few basic conventions (e.g. binding the VR headset to three.camera), it should just work. Haven't tried it yet, too much to do, but I do know these guys who have a mocap studio being repurposed for free-walking VR experiments using a wireless headset. I sat on the couch in the Unreal Engine 4 apartment two weeks ago, and picked up a mug from the coffee table (i.e. real couch/table + real mug + mocap balls attached to the mug and the headset). Magic. Would be even better with a mathbox chandelier.
I've used Mathbox some, and am still trying to get more familiar with it. Is MathBox 2 architecturally separate, or will the original Mathbox become a subset of MB2? I'll just keep plugging away at MB (the 1st).
The comparison to D3js seems apt. MathBox is -- somewhat -- a 3D version of what D3 does. But D3 takes a bring-your-own-data approach, whereas mathbox is more directly about defining the mathematical structures. Both are fairly low-level. Mathbox is more opinionated, maybe. Vega might be a more direct comparison [1].
MB2 is completely separate from v1. I replaced the tQuery dependency with Threestrap, which is much less opinionated and the opposite of monolithic. The shaders are now compiled in so it runs over file://. The API works mostly the same, only now you can nest views.
I could provide a best-effort v1 compatibility API if there is a demand for it, so you'd only need to replace your initialization code and e.g. call mathbox.v1() to get the old API. I don't know many people using MB1 though.
With regards to D3, I actually see it as quite complementary to MathBox 2. Take away all the DOM/SVG wrangling and you are left with tons of useful components, like all the geospatial stuff, for which MathBox can be the output layer. You don't actually have to use live expressions or GLSL transforms, you can just pass in a float array or a regular array of numbers, even a nested one.
Thanks for the comments, and for the immense amount of work you've put into building this. I wouldn't care much if it didn't support MB1, just wanted to understand that relationship.
As I thought about it more after posting, I imagined what you describe -- feeding in data sets (via an internal REST interface, say) and figured that would be simple enough.
I'm most interested in the multi-viewport idea, which I imagine is related to nested views. Presumably it lets you define linked representations of the same structures? Linked in the sense of brushing-and-linking [1]. I'm curious to try building some linked representations of real- and phase-space diagrams.
> Please view in Chrome or Firefox. Chrome is glitchy, Firefox is stuttery.
I really want to get behind WebGL, but when is it going to have decent performance/compatibility? I tried this out in both FF and Chrome on a powerful desktop computer (i5-4670K, GTX760, 16GB RAM) and it was glitchy/stuttery as described. Firefox rendered some scenes at what seemed like 2-3 FPS. Chrome was much smoother, but I couldn't tell what parts were glitches. For example, the "classic demoscene water effect" looked completely different in Chrome. But neither FF nor Chrome produced an effect remotely resembling water.
Although this looks like a great library, personally I prefer to stick with OpenGL programming until WebGL's quircks are sorted out.
... when the GPU drivers fix their bugs, when all the undefined behaviour gets defined, when the security holes get patched, and triple-A gaming studios with intimate connections to the vendors aren't the only ones driving the tech. Any day now.
WebGL devs are playing the longest game of chicken seen on the web yet.
Runs OK on my Surface Pro 2. Core i5, Intel GMA, 4GB of shared RAM.
There were some places where the framerate dropped but they were the more complex demos. The fan kicked in almost immediately though. In general, even if the framerate was low it was stable.
Interestingly IE11 seemed to render almost as well as Chrome and I didn't notice a speed difference. I guess that's what happens if you offload work to the GPU.
I viewed this whole presentation on a MacBook Air plugged into a 32" monitor, and while 1 or 2 of the slides would pause here and there, overall it was amazingly smooth. Mind blown.
I viewed this whole presentation on a MacBook Air plugged into a 32" monitor, and while 1 or 2 of the slides would pause here and there, overall it was amazingly smooth. Mind blown.
I was expecting another disappointment but no, on this somewhat dated system it runs great, and I was really impressed with several of the demos, even manipulating/rotating them in real time with the mouse, all running quite smoothly: Chromium Version 36.0.1985.125 Ubuntu 14.04 (283153), i5 2.8 Ghz, 6 Gb RAM
This is still emerging tech but it's stabilizing rapidly — both Firefox and Chrome play back at 60fps on a MacBook Air. On a sub-$1K laptop it's noticeably slower because things are disabled due to Intel HD 3000 driver bugs but those bugs are now getting a lot more attention since they're so visible. I'd expect the situation to be a lot more viable in a year or two, so now seems to be a good time to start learning the technology and thinking about how to use it.
I have your exact hardware configuration and all his slides got a solid 60FPS. I'm curious what your setup was. What version of Firefox? What version of Windows? Latest NVidia drivers? Any weird extensions installed, screen/desktop capture apps running?
92 comments
[ 4.6 ms ] story [ 155 ms ] threadNot all browsers support the feature though (check the MAX_VERTEX_TEXTURE_IMAGE_UNITS constant). Mobile devices could be problematic too since most (if not all) OpenGL ES 2.0-era devices don't support it in hardware.
Still this is one of the most impressive WebGL demos I've seen. Fantastic stuff.
In proper OpenGL, you'd be able to use transform feedback to write in to buffers with no loss of precision. And using buffers is less limited than texture fetches in the vertex pipeline.
For applications where precision matters (ie. everything scientific), WebGL on GLES2 devices is a no-go. WebGL standardization should pick up the pace to better match the development of OpenGL.
This was probably to enable WebGL on mobile devices that would have otherwise been locked out, but it heavily restricted things on the desktop which for the most part would have OpenGL 4 capable GPUs these days.
However given that WebGL on mobile still mostly sucks anyway, not sure if going for the lowest common denominator was the right decision.
And WebGL 1 has taken this long to reach mostly-working in implementations, it would have probably died in the crib if it had targeted the nascent GLES 3 feature set.
Running GLES shaders safely and reasonably fast in a sandbox (on top of insecure & crash prone drivers) is high wizadry.
Latest and greatest for mobile yes but the desktop world was already on OpenGL 4 at that point.
My whole point was that they could have just ignored mobile and delivered a much more powerful WebGL based on OpenGL 4 instead.
Your wish is granted: WebGL 2 draft supports transform feedback. http://www.khronos.org/registry/webgl/specs/latest/2.0/#3.5
(See my other reply about problems tracking latest GLES tightly)
Laptop or mobile? How much time did you spend watching the examples? What is your battery's storage capacity (milliamp-hours, usage hours, etc.)? Does it have a GPU? CPU usage? Is that battery drain consistent w/ other things that use that amount of CPU? etc.
Of course you're pushing 4x as many pixels as me.
Try Safari (you have to enable WebGL in the Develop menu). I wonder it's a Chrome issue.
I wonder what it needs to handle text presentation and input. HTML overlays are mentioned. Perhaps there are already WebGL text renderers that could be integrated. Of course visualizations this complex make my Macbook scream, but that's all right since I'm seeing something new (in a browser) and delightful. I have a few million data points that could benefit from vantage point like this, which need complex dependencies and controls.
Have you thought of ways around the path dependence[1] on monospace imposed by existing bodies of textmode UIs (and source code)? It seems unlikely that a new terminal-esque tool would succeed without some kind of legacy support. The best concept I've come up so far with is to build in affordances which handle legacy vs. new-world user interaction and app I/O models.
Related, I continue to hold out (vain) hope that elastic tabstops[2] will someday gain traction.
[1] https://en.wikipedia.org/wiki/Path_dependence
[2] http://nickgravgaard.com/elastictabstops/
One of the things I discovered was just how much legacy cruft is really around. Not just things like ANSI colors, but e.g. grotty syntax. It made no sense until I realized it was created for teletype printers... it underlines things by backspacing after every character and printing a "_". It bolds by backspacing and repeating the character. I had to parse this to support man pages, and I assume the default TTY still does too.
The other thing was that so much of Unix workflow really only works by accident. The fact that you can ssh + sudo + ssh + ... is because the pipes are too dumb to fuck it up. Take for example SSH escape sequences... [1] they only work on the first hop. The proper solution is out-of-band signaling.
From an architecture point of view, the whole termcaps / stdio thing is crazy. The Unix principle is supposed to be about simple agnostic composition, and yet most tools have to sniff out their environment in order to maintain this illusion. Text files are for people, not machines. And if you want to see a never ending discussion, just ask a bunch of greybeards how to write a shell script that can handle files with spaces in their name.
[1] http://lonesysadmin.net/2011/11/08/ssh-escape-sequences-aka-...
Actually, it's less that interprets this and converts it into the appropriate terminal formatting, not the TTY itself.
It's eye candy AND it's interesting at it's core... wow. Beautiful work.
I just can't articulate a better thing than "wow". Really. This is incredible.
First is that you can write vertex shaders in a reactive DOM. That makes it much easier to get pictures up on the screen. If any of you have ever messed around with vertex shaders, it can be a bit of a nuisance.
Second is that while the reactive DOM doesn't really exist as XML, it can be expressed as such, and would be easily diffable. This is important for collaboration.
Lastly, because it's making the GPU do all the work, data visualizations can be done by pushing large amounts of data to it. We should be able to see more patterns from data as a result.
Calculate just the points to be drawn, then draw them (explicit generation).
Calculate the entire surface/volume, and draw values where they exist (or based on magnitude or whatever properties are used) (implicit generation).
The second method is in some circumstances less efficient, especially if the graph is very simple and takes up little screen space, but overall much easier to work with. Its similar to the difference between ray casting and rasterization, in a way.
So if you wanted to render an implicit surface this way, you could do e.g. marching cubes or tetrahedra on a grid, and only feed in a scalar 3D field, either as an array or as a procedural function. Or you could do a <raymarch> operator for raymarching a distance field. On the inside, this could be a dumb per-pixel loop, or do recursive quad-tree subdivision. You shouldn't need to care.
It's all vaporware right now, but it's just a matter of fitting it in neatly.
Nice.
First time I heard about Steven was when I saw this [2] post last year.. the best part is that he leaves many easter eggs or "achievements" around for you to discover :)
[1] acko.net [2] https://news.ycombinator.com/item?id=6268610
These is no need to make this into a pissing contest or a rivalry. We're fans of Steven's work and incredibly impressed with how he has pushed the state of the art on the web forward. Anyone who works on the bleeding edge like this helps build a brighter future for the web and creates more knowledge upon which others may build. Anyways, please keep the discussion focused on what Steven has achieved here instead of trolling.
Steven, many kudos for this. Extraordinary work.
[0] http://acko.net/blog/shadow-dom/
[1] http://extensiblewebmanifesto.org/
http://test.co.s3.amazonaws.com/Screen%20Shot%202014-08-15%2...
I have waited so long for a good hardware-accelerated 3D screensaver in my browser! ;)
A discussion group, sometimes informal, interested in a particular topic.
Conferences often refer to their themed tracks as "BoF" sessions.
Now I hate 2D screens even more. So yes.
https://github.com/unconed/threestrap http://thesawmill.ca/ http://wavesine.com/
The comparison to D3js seems apt. MathBox is -- somewhat -- a 3D version of what D3 does. But D3 takes a bring-your-own-data approach, whereas mathbox is more directly about defining the mathematical structures. Both are fairly low-level. Mathbox is more opinionated, maybe. Vega might be a more direct comparison [1].
[1]: https://github.com/trifacta/vega/wiki/Vega-and-D3
I could provide a best-effort v1 compatibility API if there is a demand for it, so you'd only need to replace your initialization code and e.g. call mathbox.v1() to get the old API. I don't know many people using MB1 though.
With regards to D3, I actually see it as quite complementary to MathBox 2. Take away all the DOM/SVG wrangling and you are left with tons of useful components, like all the geospatial stuff, for which MathBox can be the output layer. You don't actually have to use live expressions or GLSL transforms, you can just pass in a float array or a regular array of numbers, even a nested one.
As I thought about it more after posting, I imagined what you describe -- feeding in data sets (via an internal REST interface, say) and figured that would be simple enough.
I'm most interested in the multi-viewport idea, which I imagine is related to nested views. Presumably it lets you define linked representations of the same structures? Linked in the sense of brushing-and-linking [1]. I'm curious to try building some linked representations of real- and phase-space diagrams.
[1]: http://bl.ocks.org/mbostock/4063663
I really want to get behind WebGL, but when is it going to have decent performance/compatibility? I tried this out in both FF and Chrome on a powerful desktop computer (i5-4670K, GTX760, 16GB RAM) and it was glitchy/stuttery as described. Firefox rendered some scenes at what seemed like 2-3 FPS. Chrome was much smoother, but I couldn't tell what parts were glitches. For example, the "classic demoscene water effect" looked completely different in Chrome. But neither FF nor Chrome produced an effect remotely resembling water.
Although this looks like a great library, personally I prefer to stick with OpenGL programming until WebGL's quircks are sorted out.
WebGL devs are playing the longest game of chicken seen on the web yet.
There were some places where the framerate dropped but they were the more complex demos. The fan kicked in almost immediately though. In general, even if the framerate was low it was stable.
Interestingly IE11 seemed to render almost as well as Chrome and I didn't notice a speed difference. I guess that's what happens if you offload work to the GPU.
I don't get why he says vertex shaders aren't doable in web GL though. Don't the various shadertoy type sites let you write vertex shaders right now?