Howdy HN audience, wanted to share a revamped proof-of-concept that I originally wrote in 2013 for GPU accelerated text rendering in WebGL.
This isn't intended to be a real text editor prototype, just a demonstration of how much text you can push through a modern GPU if your shaders are reasonably efficient. The original PoC could handle ~800 megs of text at 60 fps on the integrated GPU of a 2012 Macbook Pro, this one has been tested with 1.2 gigs of text on a 3080ti, where it uses only a few percent of the GPU's capacity and nearly zero CPU.
And yes, I know there's no syntax highlighting or kerning or vector font support in Wideboard - that's not the point, and those features could be added without majorly affecting performance.
My hope is that this will spark a bit of discussion about text rendering and IDE design. How would our programming habits change if we could view our entire codebase - even an enormous one - in a single view ala Google Maps? I don't currently have the spare time to investigate something as sizable as an IDE project, but I'm curious to see what other folks come up with.
You can record app execution, and then play it back thread and line and by line to see each line executing in time. It’s kinda fun to see millions of lines flash and highlight and move at 60fps and seeing implicit relationships between executing files and high level functional flows
>How would our programming habits change if we could view our entire codebase - even an enormous one - in a single view ala Google Maps?
One way is to use a 3D approach to the visualization problem [1]. Primitive has some videos out showing this in practice and its still very much work in progress.
Text rendering in XR is such a neglected backwater of the UI shell although some projects like TrokiaJS are making progress. Will definitely look at your GLSL with someone more skilled in the art - thanks for sharing.
I’ve tried desperately to get in contact with the folks at primitive to hash out some ideas and make suggestions, but they’re either not open to them, or too busy to have a conversation. I’ve been developing a SceneKit and metal backed GPU based text visualizer in a fully 3D space, but without all the bells and whistles of an IDE yet - although I can trace threads and do basic syntax analysis. If that interests you, check out: https://github.com/tikimcfee/LookAtThat
I think Primitive started out with a bang under one of the many false dawns of XR and while it is still a side project, it is not super active although I did see JavaScript added in the last 6months so definitely not frozen.
Your own project looks like a great experiment with Apple ecosystem and while I hope they join on to standards based WebXR it will be good to see this new development environment they are rumored to release this summer.
I had not seen TapKit before so thats a great thing to be aware of thanks & it reminds me of a project called Lexting that was floated for a while using freespace inputs and still some demos on Youtube.
Zooming isn't working for me in Firefox with a trackpad. Any idea what's going on? It doesn't work even in the hosted version someone shared below:https://wideboard.stjo.dev/
I find this very exciting because it makes plausible that we can render 'infinite canvas' surfaces for user workspaces at high speed on current hardware (like my cheap laptop I just ran the code on), inside the browser (which makes it potentially instantly accessible across a network &cpp).
Miro has that property and I find it infuriating to use for almost any purpose. I can never find what others have drawn nearly as quickly as I can find test in a 1D document.
This I think is a very valid concern. When a canvas with stuff on it is of infinite size, then however big your finite content is, you will not be able to find it in any limited amount of time in the general case. This effect also shows itself in the Wideboard demo: you can zoom out, locate the top end of the long text, zoom in and start reading, using cursor keys to scroll up and down (even page down does work). However there's little in the way to keep you 'on the track' as it were, as in a text editor where you're either at a specific part of one document or one part of another. Meaning that what's missing here is a lot of user interface enhancements, the idea being that you can 'lock into' a given document and see it just as in a classical application, and switch to another document without having to manually zoom out, pan, zoom in. As far as I remember it, early GUIs (early Linux desktops, OS/2, GEM) had similar problems: to many fiddly windows with too many fiddly mouse-dependent controls, to much nudging windows on the screen so they just fitted. This IMO has improved, and a barebones infinite canvas is not going to work well for a vast majority of users and use cases; you do need the guardrails, and it's probably also imperative that you can always work on that canvas just like you're working on the desktop now, only with the added benefit that now you can, if you want to, spatially arrange things, get a high-level overview, build a memory where you put things spatially, use the canvas to do a layout of a paper or a book, that sort of thing. Much like the minimap that Sublime Text (and, I think, Atom and VS Code after it) show, and similar to how applications like Inkscape work today.
When I was actively developing my text editor my favorite test file was made by concatenating all .c, .h and .S files from Linux kernel, which at the time made it a 542MB file with almost 20M lines. It was funny to see how some editors struggled with it.
And then there's some modern IDEs that you'd think would be able to handle measly hundred-lines-long files, but they sometimes don't.
At $WORKPLACE I'm unfortunately required to use Visual Studio because the project won't work with anything else, and there's some specific file types (I think it's the ASAX files) that make VS hang for several seconds, where the code is literally uneditable (can't scroll, there's no syntax highlighting, and more weird stuff I'm surely forgetting).
On a bad day, those files don't even get line numbers because the whole left-side gutter is missing, and I'm not even joking.
I'm already looking to change jobs for one where the project isn't that horribly tied to a specific IDE.
cool demo. I loaded my own codebase to look around in it. I could really see this being added to an IDE to get an overview of a code base (and it also looks good).
Some tips for others (when trying with your own repo):
- you need to rename any index.html files and clear the browser cache otherwise it skips that folder
- change the value at wb-librarian.js:98 (this.screenCursorY) to a value that better matches the size of your code base
It’s really inspiring to see this come alive. I’ve been struggling with my own version of this in a 3D space, and if I may be so rude as to plug it a bit, I’m able to push just as much text into a fully interactive and highlighted view here: https://github.com/tikimcfee/LookAtThat
I hope the above shows that indeed, GPU is where it’s at!
I sometimes think about the untapped potentials of interface design or code editors in a zoomable mode like this where files, text, folders or whatever is laid out on a huge map.
This seems to mirror the way we naturally understand and traverse space. Ah "this text file is this big", "this file is to the right of this file", this project has this tree structure, has these relations to other files, this is the lifecycle of this application, this variable jumps from this to this file etc.
I feel like visualisation in 2d and 3d space in general is has not been explored enough.
One of the ideas in my "ideas book" has to do with a Git client written using ImGui. There are so many things I could see that could be better done on the visualization side of things if one could leverage the render power of a GPU.
a11y is another topic with that kind of rendering, though…
I've been thinking about a hyperbolic desktop for some time. Basically everything is visible on the screen, but the stuff you don't need is shrunken at the side. No idea if it would be useful, but would be fun to try.
It could be a really interesting way to visualize code. I could imagine using profiler output to influence the rendering of different code. Similarly, an interactive debugger view that showed e.g. objects being created, or memory usage.
Not entirely relevant, but it seems like the author is using windows 7 with an old build of chrome. Is there a reason for this? Maybe testing on an old PC to easily spot performance problems? Yet the readme mentions a Nvidia 3080ti gpu, which is quite new.
39 comments
[ 4.9 ms ] story [ 90.7 ms ] threadThis isn't intended to be a real text editor prototype, just a demonstration of how much text you can push through a modern GPU if your shaders are reasonably efficient. The original PoC could handle ~800 megs of text at 60 fps on the integrated GPU of a 2012 Macbook Pro, this one has been tested with 1.2 gigs of text on a 3080ti, where it uses only a few percent of the GPU's capacity and nearly zero CPU.
And yes, I know there's no syntax highlighting or kerning or vector font support in Wideboard - that's not the point, and those features could be added without majorly affecting performance.
My hope is that this will spark a bit of discussion about text rendering and IDE design. How would our programming habits change if we could view our entire codebase - even an enormous one - in a single view ala Google Maps? I don't currently have the spare time to investigate something as sizable as an IDE project, but I'm curious to see what other folks come up with.
-Austin
It would be neat to see each line or line-chunk as flashes that happen across every file in the Linux kernel since the first commit, all at once.
I think there was also a visualization for Minecraft source that used graphs to show the evolution of that code.
[1] https://justine.lol/blinkenlights
You can record app execution, and then play it back thread and line and by line to see each line executing in time. It’s kinda fun to see millions of lines flash and highlight and move at 60fps and seeing implicit relationships between executing files and high level functional flows
One way is to use a 3D approach to the visualization problem [1]. Primitive has some videos out showing this in practice and its still very much work in progress.
Text rendering in XR is such a neglected backwater of the UI shell although some projects like TrokiaJS are making progress. Will definitely look at your GLSL with someone more skilled in the art - thanks for sharing.
[1] https://primitive.io
Your own project looks like a great experiment with Apple ecosystem and while I hope they join on to standards based WebXR it will be good to see this new development environment they are rumored to release this summer.
I had not seen TapKit before so thats a great thing to be aware of thanks & it reminds me of a project called Lexting that was floated for a while using freespace inputs and still some demos on Youtube.
Good luck with your prototyping!
Seeing all that text on one screen really puts the size of GPU memory in perspective in a way that texture data doesn’t.
At $WORKPLACE I'm unfortunately required to use Visual Studio because the project won't work with anything else, and there's some specific file types (I think it's the ASAX files) that make VS hang for several seconds, where the code is literally uneditable (can't scroll, there's no syntax highlighting, and more weird stuff I'm surely forgetting).
On a bad day, those files don't even get line numbers because the whole left-side gutter is missing, and I'm not even joking.
I'm already looking to change jobs for one where the project isn't that horribly tied to a specific IDE.
It reminds me of the following demo: https://wdobbie.com/pdf/ from this post: https://wdobbie.com/post/gpu-text-rendering-with-vector-text...
Some tips for others (when trying with your own repo): - you need to rename any index.html files and clear the browser cache otherwise it skips that folder - change the value at wb-librarian.js:98 (this.screenCursorY) to a value that better matches the size of your code base
That being said, does anybody know how much GPU RAM does it actually require? Would 6GB suffice?
I hope the above shows that indeed, GPU is where it’s at!
I sometimes think about the untapped potentials of interface design or code editors in a zoomable mode like this where files, text, folders or whatever is laid out on a huge map.
This seems to mirror the way we naturally understand and traverse space. Ah "this text file is this big", "this file is to the right of this file", this project has this tree structure, has these relations to other files, this is the lifecycle of this application, this variable jumps from this to this file etc.
I feel like visualisation in 2d and 3d space in general is has not been explored enough.
It’s runnable if you’re in macOS.
a11y is another topic with that kind of rendering, though…
But it would be harder to write, and add geometric relations between windows that might not be so useful.
And you can run a “modern” (1999) recreation: https://en.m.wikipedia.org/wiki/File_System_Visualizer
If it ran at 60fps on a 2012 Macbook pro, I have to say I'm very surprised by how poorly it scales