gamdevs have been waiting for this for something like a decade. I'd like to know if anything has changed to expect it to land anytime soon esp in Safari.
OffscreenCanvas isn't supported by Firefox yet. Also, if a canvas is just not attached to display it can act like offscreen canvas. Drawing on it will be much faster than on screen canvas. When you are done drawing, take the image data from offscreen canvas and draw on the live canvas.
The one major stumbling block not mentioned in the article is accessibility. OffscreenCanvas offers nothing to solving that problem.
Also, I don't understand people who complain that 2d canvas is slow; it isn't. What slows things down is poor planning and execution of the animation. For instance - does the entire Mandlebrot Set need to be calculated on every RequestAnimationFrame iteration? If I was tackling the problem I'd only calculate the data when one of the significant parameters changed - in a worker farm, wasm, whatever - and only update the display canvas when the results emerged.
My canvas library[1] relies heavily in normal canvases not attached to the DOM. Not only does it speed things up massively, it also allows the library to do things like make the canvas responsive[2].
Canvas is one of the only web APIs that's simple enough to understand in one reading, yet flexible enough to be a real application primitive — and that's because it wasn't originally designed for the web, but was simply a lightweight wrapper around CoreGraphics for Apple's Dashboard in Mac OS X.
Maybe Apple should have provided an NSView wrapper too, with a drawRect() method to draw into a canvas, and made DOM elements part of that view tree. That would have been a much simpler API for web components than what the committees came up with.
Note that OffscreenCanvas is currently only implemented in chromium based browsers [0]
It is a neat idea, basically separating all the canvas calls into a webworker so it can run on it's own thread. However this scale of parallelism is not a solution to fundamentally slow code... the map rendering performance is still poor, it's just been prevented from interfering with the main thread. Please don't throw web workers at stuff before asking why the code or API calls are slow first.
The GP meant that a fundamentally slow code can't be made faster that much. Offloading canvas calls without making the rendering itself faster can easily mask the performance issue, which is very relevant (and frequently ignored) in mobiles.
There are definitely legitimate use cases, like rendering text tiles in parallel workers and letting the main thread blit them all together. Text rendering is one of the slowest canvas drawing operations, and blitting is super fast, so this can make a huge difference in responsiveness if you’re zooming or scrolling.
Without off screen canvas and threading your UI and logic are all being done in a single thread, causing significant lag problems that every other kind of application has resolved since the 90's. Threading is am essential part of any user facing application, and browsers outside of chrome have continually squandered the ability to put out good quality UI that does intensive background work. Something as simple as a video player in wasm requires at least 1 other thread.
There's still a major disconnect in this blog post. The problem statement was making all web-enabled devices performant with Canvas, not just desktops and mobiles running Chrome. If Off Screen Canvas is only available for a recent version of one browser engine, it's not going to solve that problem is it?
The core map code is slow, why doesn’t the map code optimize its runtime by using threads? Then the main thread canvas just renders the current state? That’s how you solve the problem, not by creating an entire new canvas to solve the crappy map code. You’re right, people have solved these problems with threads, but on a much better way. Google needs to stop reinventing the wheel and fix their problems.
I don't understand. Was it not possible to decouple the UI/events thread from whatever performance intensive things your app might be doing? We've had HTML5 games for a long time now and they've figured this out.
Is this just a batteries-included way of forcing the thread separation between logic and drawing?
There is a "best practices" way to do canvas drawing which involves requestAnimationFrame inside a draw loop function, but there is no way to put UI events and drawing on different logical threads. The best you can do is make sure your draw function does not do non-drawing things.
In theory it's possible to break up long running operations into small chunks and schedule them between frames. In practice it's a lot of work and the only people who bother are game developers. Most app developers just don't care at all if they have a 100ms pause. If you point it out to them they'll just tell you that 100ms is nothing and nobody will notice, they have better things to do than fix 100ms pauses. And so all apps are janky.
> Is this just a batteries-included way of forcing the thread separation between logic and drawing?
It's not possible to draw canvases on multiple threads without this. So it's about enabling, not just forcing. Enabling multiple threads is a big deal because every device has multiple cores these days and the web is mostly using only one.
Expanding on this, if you do all the work on the main thread, you're implementing a cooperative multitasking system, with all of the problems that entails.
It's surprisingly hard to break up potentially long tasks in an ergonomic way without losing a ton of performance, tracking down a long tail of tasks that block rendering, or both. Having multiple threads of execution is so much better.
> What's interesting though, is that in reality, <canvas>'s prevalence in the the HTTPArchive isn't so far behind <video> (63rd/70th most popular elements respectively). It's considerably more widely used than many other standard HTML elements.
> Amazing, right? I mean, how could that even be?!
> The short answer is, it's just harder to recognize. A great example of this is maps. As a user, you recognize maps. You know they are common and popular. But what perhaps you don't recognize that it's on a canvas.
I’d posit that the majority of canvases in the wild is tracking and fingerprinting, not something that ends up visible to the user.
Unfortunately, privacy-conscious users are best off blocking canvas by default, like Tor Browser does.
You are almost certainly wrong. I've seen canvas used on many pages for rendering documents. Some frameworks render a majority of the UI in a canvas. Games are often in a canvas.
Fingerprinting and tracking done using many features. Canvas is one but not the only. Blocking canvas probably makes it easier to fingerprint you.
The linked video demo is really impressive https://youtu.be/8aAdP9nQSss?t=303 (rendering a mandelbrot from barely usable to fully responsive and animated)
I have been disappointed by cross browser discrepancies in the canvas element. Chrome and FF handle Alpha values very differently, and the results are extremely noticeable if you do anything that involves layering many semi-transparent layers on top of each other. There are other floating point rounding error quirks as well, e.g., firefox can translate a context by 0.001px whereas chrome is limited to a precision of only 0.01px. These discrepancies make it impossible to do certain things that look the same on all browsers.
Back in the day we used to call that "double buffering".
The common Amiga implementation was especially neat. There, the video buffer was (almost) any chunk of memory you pointed the "screen" at. You could pre-render a number of complex frames, and have a high-FPS animation by moving the buffer pointer.
And yes, you could look at your own code executing in realtime. But I digress.
From my understanding, OffscreenCanvas and double buffering are unrelated. OffscreenCanvas just lets you execute draw calls in a separate thread (in a WebWorker). I believe the browser already uses double buffering under the hood for drawing to a canvas element.
The biggest performance hit that is educated everywhere is accessing pixels through getImageData() as bytes. In many cases it is way better to access it as a 32-bits RGBA value, like with:
const rgba = new Uint32Array(getImageData(0, 0, width, height).data.buffer);
rgba[y * width + x] = R | G << 8 | B << 16 | A << 24;
And using Array.subarray as substitute for memcpy() for block copy:
Seeing canvas used always makes me sad. Why can’t maps use SVG? If there are blockers for that, I’d rather see the web resolve those. canvas is a black box for screen readers, crawlers, the semantic web. You end up reimplementing a DOM anyway for accessibility, or not, if you don’t care about accessibility, which sucks.
I guess it would be fair to ask if canvas can eventually become something to meet those challenges. Not by itself obviously but when you consider things like Flutter built on top of it. It’s not there yet obviously but I could totally see a path in the future where it handles all those use cases just fine and do so with better performance.
I've been part of an implementation of a global map done in SVG. Only path data that was sent down was national borders. We had to do many multiple levels because the volume of data to get the paths drawn accurately was both too big and too slow until you zoomed in a bit and could start loading individual countries, then individual regions.
Rendering rasterized tiles from a tile server onto a canvas means that, on the client, performance is identical regardless of zoom level.
Basically, we went with SVG because only drawing the borders looked cool, and we could fake some clever zoom animations, and we couldn't afford to purchase custom tile data and run a tile server.
As for interactive bits, those are frequently done with DOM nodes positioned over the map. Maps make for a pretty crap user experience for screen readers, so if you really wanted to make the information accessible, you're better off putting it in a separate interface. That, and I can honestly say I have yet to see a map that would make sense to have a crawler attempt to index.
Good time to be a Flutter developer. I’m really excited to see where they can get on web in particular in the next couple of years but this will certainly help.
43 comments
[ 404 ms ] story [ 2574 ms ] threadhttps://caniuse.com/offscreencanvas
The one major stumbling block not mentioned in the article is accessibility. OffscreenCanvas offers nothing to solving that problem.
Also, I don't understand people who complain that 2d canvas is slow; it isn't. What slows things down is poor planning and execution of the animation. For instance - does the entire Mandlebrot Set need to be calculated on every RequestAnimationFrame iteration? If I was tackling the problem I'd only calculate the data when one of the significant parameters changed - in a worker farm, wasm, whatever - and only update the display canvas when the results emerged.
My canvas library[1] relies heavily in normal canvases not attached to the DOM. Not only does it speed things up massively, it also allows the library to do things like make the canvas responsive[2].
[1] - Scrawl-canvas - https://scrawl-v8.rikweb.org.uk/
[2] - CodePen demo of a responsive canvas - https://codepen.io/kaliedarik/pen/jOmWwWy
Maybe Apple should have provided an NSView wrapper too, with a drawRect() method to draw into a canvas, and made DOM elements part of that view tree. That would have been a much simpler API for web components than what the committees came up with.
It is a neat idea, basically separating all the canvas calls into a webworker so it can run on it's own thread. However this scale of parallelism is not a solution to fundamentally slow code... the map rendering performance is still poor, it's just been prevented from interfering with the main thread. Please don't throw web workers at stuff before asking why the code or API calls are slow first.
[0] https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCa...
[1] https://www.khronos.org/opengl/wiki/Default_Framebuffer#Doub...
Is this just a batteries-included way of forcing the thread separation between logic and drawing?
> Is this just a batteries-included way of forcing the thread separation between logic and drawing?
It's not possible to draw canvases on multiple threads without this. So it's about enabling, not just forcing. Enabling multiple threads is a big deal because every device has multiple cores these days and the web is mostly using only one.
It's surprisingly hard to break up potentially long tasks in an ergonomic way without losing a ton of performance, tracking down a long tail of tasks that block rendering, or both. Having multiple threads of execution is so much better.
> Amazing, right? I mean, how could that even be?!
> The short answer is, it's just harder to recognize. A great example of this is maps. As a user, you recognize maps. You know they are common and popular. But what perhaps you don't recognize that it's on a canvas.
I’d posit that the majority of canvases in the wild is tracking and fingerprinting, not something that ends up visible to the user.
Unfortunately, privacy-conscious users are best off blocking canvas by default, like Tor Browser does.
https://en.wikipedia.org/wiki/Canvas_fingerprinting
Fingerprinting and tracking done using many features. Canvas is one but not the only. Blocking canvas probably makes it easier to fingerprint you.
Google's Recaptcha uses canvas for fingerprinting, for one.
Does disabling canvas disable recaptcha's fingerprinting? Or does it make it more effective?
The common Amiga implementation was especially neat. There, the video buffer was (almost) any chunk of memory you pointed the "screen" at. You could pre-render a number of complex frames, and have a high-FPS animation by moving the buffer pointer.
And yes, you could look at your own code executing in realtime. But I digress.
https://rockingship.github.io/jsFractalZoom/jsFractalZoom.ht...
It can easily handle 4K resolution.
Rendering rasterized tiles from a tile server onto a canvas means that, on the client, performance is identical regardless of zoom level.
Basically, we went with SVG because only drawing the borders looked cool, and we could fake some clever zoom animations, and we couldn't afford to purchase custom tile data and run a tile server.
As for interactive bits, those are frequently done with DOM nodes positioned over the map. Maps make for a pretty crap user experience for screen readers, so if you really wanted to make the information accessible, you're better off putting it in a separate interface. That, and I can honestly say I have yet to see a map that would make sense to have a crawler attempt to index.