The title on HN is very editorialized and I can't overemphasize how preliminary and experimental this work is. Take all numbers with a huge grain of salt.
That said, I'm obviously really excited about this project. :) I feel strongly that it's the right from-scratch, modern approach for painting Web content. Happy to answer any questions.
Glenn Watson deserves nearly all the credit here. My role has been limited to a little bit of high-level design input and messing around with signed distance fields and goofy stencil buffer tricks. :)
Ooooh signed distance fields, if you ever get a good Rust implementation going make sure to publish it and add a readme that at least says what the code is so that it can be found.
It could be great for other rust libraries, apps and games. None of them have subpixel rendering at the moment AFAIK and that's super easy with distance fields and real important for appearance.
>None of them have subpixel rendering at the moment AFAIK
Actually, most Rust projects end up using Freetype for glyph rendering, which does support subpixel rendering.
Signed distance fields are great for animated text, but they also have a couple of disadvantages - they generally produce lower quality output than Freetype at small glyph sizes, and the signed distance field is more complex to generate at start, which slows down initial page rendering. Given that 99% of web pages are static, it's a harder sell. Maybe some sort of heuristic based switching would work.
I'm surprised to hear they produce lower quality output at small glyph sizes. Fragment shaders can do multiple texture samples and I've gotten very high quality results. You can even do ClearType-style subpixel rendering with a handful of samples. The case where signed distance fields don't work is for large glyph sizes. They often end up chopping off corners or not having the density to represent thin calligraphic strokes.
> What does the GPU accelerate? I actually pretty surprised that there is anything that can be parallelized to that level.
Essentially everything but font rasterization (assuming you don't count image decode as part of painting). The major trick here is to avoid vector graphics, since you don't need a full-blown vector graphics library to paint common CSS.
> signed distance field font rendering? Nice
Note that this isn't integrated yet; it's a pile of experimental code, and I have some lingering doubts about its performance. But I would love it if we (or someone else!) can make it work!
Eh, I'm skeptical. :) IMO, SVG has serious ergonomic and performance issues that make it unlikely that we'll see a large-scale transition to it anytime soon. Not to mention that vector graphics are not really what you want for mobile apps: the proven framework is something like Core Animation, which CSS is much closer to than SVG.
But I don't really want to debate this here, as it's somewhat off-topic. And there are only so many ways to draw a button, SVG or not. WebRender is really good at drawing buttons. So there's a possibility that we could use the same technique, just extended, for SVG.
Do you have any good comparison numbers from Firefox or Chrome? The numbers sure seem impressive when compared to Servo but I know that Chrome has some pretty awesome rendering infrastructure and I'm wondering how this compares.
There was one screenshot of Firefox included but it seemed to cherry pick the longest paint and didn't offer any explanation of how to compare it or which Servo+Webrender numbers to compare it to.
There have been some really back-of-the-envelope comparisons with Skia-GL, but nothing rigorous enough to write up. That's one major reason why the numbers should be taken with a grain of salt.
Servo uses the same painting backend as Firefox does: Moz2D (also known as Azure). Moz2D is itself an abstraction layer over Cairo, Skia, Core Graphics, and Direct2D. So comparing against current Servo is actually a good way to compare against Firefox and Chrome in CPU painting mode: it's using the same painting backend as both of those browsers and eliminates any noise that results from layout differences.
The most important thing to note is that the overall approach is quite different from Skia-GL (Ganesh). Skia is largely an immediate mode API (with a few retained-mode-style optimizations available with SkPictures, but the overall approach is immediate). WebRender (preliminary name), however, is a fully retained mode API and has no immediate mode at all, allowing it to focus on aggressive optimizations like "strength-reducing" clipping regions and Z-buffering opaque content for improved batching. This is a tradeoff, and it'd be easy to construct pathological cases in which one API or the other performs better. What we need to find out is which wins on real-world content, and while our early results are promising (and intuitively it seems highly probable that a retained mode API will win for CSS and the DOM), there's a lot more investigation that needs to happen before it'll be a slam-dunk case.
> Moz2D is itself an abstraction layer over Cairo, Skia, Core Graphics, and Direct2D.
Out of curiosity, given that both Cairo and Skia run on all platforms Firefox does and have GPU-accelerated backends, why abstract over more than one of them rather than picking one and optimizing it for all platforms?
For Cairo, it's basically for legacy reasons, as well as looking the same as GTK+ (which I don't think is particularly important, but reasonable people can differ). Direct2D, however, has historically been the best-of-breed vector graphics engine, outclassing the rest by large margins, so it's definitely worth it to use it on Windows where available. Skia-GL has caught up quite a bit with the Ganesh effort, but Direct2D has historically been so much more mature and continues to get better with every Windows release, so I think it makes sense to keep the abstraction layer around.
I hope you keep use without servo in mind at least somewhat - with the right kind of license this kind of library could be a nice foundation for a GUI framework.
We still need a headless GL compositor even today to pass the reftests. The only way to composite layers together is GL. (This will probably change when we support printing though.)
> The only way to composite layers together is GL. (This will probably change when we support printing though.)
If not for the (very reasonable) expectation that things like "print to PDF" should produce vectors and selectable text rather than raster images of pages, you could render pages for printing using GL, too.
(Though this is somewhat belated--since it took 4 days, I suspect you'll never see this. I'd be interested to know if I'm wrong; please speak up in that case!)
I'd love to see a Web renderer based on Vulkan. The reduced CPU overhead and much improved threading should be a huge win. You'd be able to cache on the command buffer level instead of just the vertex buffer level. Seems like a bleeding edge graphics API would be a good match for an experimental project like Servo as well.
To really use Vulkan as efficiently as possible, though, you'd probably want to write a renderer mostly from scratch rather than maintaining some sort of abstraction layer to support both GL and Vulkan.
I agree! The current graphics and driver landscape is too fragmented to bet on Vulkan in the near-term, however.
We do anticipate being able to keep most of the code around when Vulkan hits the scene. Our approach isn't to write an abstraction layer per se but rather by keeping our direct GL usage as isolated as possible, so that we can easily rewrite that stuff from scratch to really make good use of Vulkan, as you say. (This also allows us to support D3D if we want to.) Most of the logic here—batching, culling, texture atlas management, display list optimization—is very general logic that will apply no matter what hardware API we end up using.
"Near term" is relative. Multicore has been here for many years and we can safely assume its presence (though even then we're making sure we have good sequential performance). But it's not even clear that Vulkan will be available at all on all devices we might care about in the foreseeable future.
I often run browsers over ssh and I really hope this "draw programs like a game engine" stuff doesn't catch on.
This is already the approach taken by Chrome (trace Chrome and you'll just see a bunch of full window resolution PutImage calls), and it's very slow over the network, I get around 1 fps.
Firefox on the other hand uses more X11 calls, and it's much more responsive, close to running it locally.
I'd call this more a gap in modern remoting functionality on Linux than anything else. With the prevalence of hardware encoding and decoding, there is little reason not to just send H.264 (or your choice of video codec) over the network. Not (just) to compensate for the kind of issue you mention, but also because it allows animations, videos, games, etc. to work over the network, and is likely to compress better than X11 anyway.
Actually, I believe NX already does this in the latest (proprietary) version, though I don't use it.
But video codecs does not give the same experience as running apps that are actually written to take some care about what/how it updates. Not least because the artefacts when they can't keep up are less consistent with what I at least expect from a desktop environment.
Apart from the fact that if nothing changes, it shouldn't send anything. I don't feel like burning 100k/s in data just to receive a pixelated image that doesn't change.
If you care about performance, why not run Firefox/Chrome locally using SSH's dynamic proxy mode? The slight hit in network latency should be much lower than the benefit of doing all of the decoding & rendering locally.
> Firefox on the other hand uses more X11 calls, and it's much more responsive, close to running it locally.
I guess this is referring to XRender. XRender might well be the right approach in theory. Unfortunately, XRender is pretty much a failed experiment as far as browsers are concerned. There was a lot of noise in the Linux community demanding its use, which is why it went into Firefox. But ultimately what people want is fast vector graphics in practice, which you can only get by tightly controlling the vector graphics stack. The reality on the ground is that you can't trust the X server to do vector graphics efficiently. The X11-remoting use case is a tiny, tiny niche that unfortunately simply isn't worth optimizing for, especially with Wayland around the corner.
Is there an explanation of how exactly this differs from current browsers?
All things that are described in the page seem pretty "normal" and "obvious" things to do, so it's not quite clear which things are not being done in current browsers.
The main differences, as I explained in another comment, are that (a) the API is fully retained-mode, not immediate-mode; (b) CSS is rendered directly without going through a vector graphics API first. The former feature allows us to achieve very aggressive batching and GL state optimizations, since we always know we have the entire draw list available to us. The latter feature has two benefits: (1) it makes all these optimizations much easier to do, since there are no vector paths involved and clipping regions are much simpler; (2) it allows us to skip all the difficult, slow stuff that arbitrary paths require (CPU path rendering, complex tessellation, stencil buffer clipping, etc.) in favor of simple hand-optimized code that renders CSS directly.
At first I thought it might be a good idea to implement a gallium statetracker for such a framework but with the direct3d statetracker not being supported by projects like wine I thought maybe vulkan will open up an opportunity to implement something with similar performance which is driver,vendor and platform agnostic.
I think Skia-GL might (and who knows what Direct2D does).
Note that one of the biggest benefits of WebRender is that it doesn't need to do any of that: if you render CSS directly, you can avoid all of these complex path rendering techniques.
49 comments
[ 3.6 ms ] story [ 103 ms ] threadThat said, I'm obviously really excited about this project. :) I feel strongly that it's the right from-scratch, modern approach for painting Web content. Happy to answer any questions.
It could be great for other rust libraries, apps and games. None of them have subpixel rendering at the moment AFAIK and that's super easy with distance fields and real important for appearance.
Signed distance fields are great for animated text, but they also have a couple of disadvantages - they generally produce lower quality output than Freetype at small glyph sizes, and the signed distance field is more complex to generate at start, which slows down initial page rendering. Given that 99% of web pages are static, it's a harder sell. Maybe some sort of heuristic based switching would work.
Edit: signed distance field font rendering? Nice
Essentially everything but font rasterization (assuming you don't count image decode as part of painting). The major trick here is to avoid vector graphics, since you don't need a full-blown vector graphics library to paint common CSS.
> signed distance field font rendering? Nice
Note that this isn't integrated yet; it's a pile of experimental code, and I have some lingering doubts about its performance. But I would love it if we (or someone else!) can make it work!
You're saying that WebRenderer should work for faster for anything but vector graphics.
Is there a risk that it's optimizing for todays/yesterdays use cases?
But I don't really want to debate this here, as it's somewhat off-topic. And there are only so many ways to draw a button, SVG or not. WebRender is really good at drawing buttons. So there's a possibility that we could use the same technique, just extended, for SVG.
There was one screenshot of Firefox included but it seemed to cherry pick the longest paint and didn't offer any explanation of how to compare it or which Servo+Webrender numbers to compare it to.
Servo uses the same painting backend as Firefox does: Moz2D (also known as Azure). Moz2D is itself an abstraction layer over Cairo, Skia, Core Graphics, and Direct2D. So comparing against current Servo is actually a good way to compare against Firefox and Chrome in CPU painting mode: it's using the same painting backend as both of those browsers and eliminates any noise that results from layout differences.
The most important thing to note is that the overall approach is quite different from Skia-GL (Ganesh). Skia is largely an immediate mode API (with a few retained-mode-style optimizations available with SkPictures, but the overall approach is immediate). WebRender (preliminary name), however, is a fully retained mode API and has no immediate mode at all, allowing it to focus on aggressive optimizations like "strength-reducing" clipping regions and Z-buffering opaque content for improved batching. This is a tradeoff, and it'd be easy to construct pathological cases in which one API or the other performs better. What we need to find out is which wins on real-world content, and while our early results are promising (and intuitively it seems highly probable that a retained mode API will win for CSS and the DOM), there's a lot more investigation that needs to happen before it'll be a slam-dunk case.
Out of curiosity, given that both Cairo and Skia run on all platforms Firefox does and have GPU-accelerated backends, why abstract over more than one of them rather than picking one and optimizing it for all platforms?
Switching to only GPU I think implies the need for at least a reference GPU backend such as mesa.
If not for the (very reasonable) expectation that things like "print to PDF" should produce vectors and selectable text rather than raster images of pages, you could render pages for printing using GL, too.
Indeed. Care to suggest one that's accurate and neutral? (In the meantime we'll take out the word "impressive".)
Congratulations on the ongoing work, too!
(Though this is somewhat belated--since it took 4 days, I suspect you'll never see this. I'd be interested to know if I'm wrong; please speak up in that case!)
To really use Vulkan as efficiently as possible, though, you'd probably want to write a renderer mostly from scratch rather than maintaining some sort of abstraction layer to support both GL and Vulkan.
I agree! The current graphics and driver landscape is too fragmented to bet on Vulkan in the near-term, however.
We do anticipate being able to keep most of the code around when Vulkan hits the scene. Our approach isn't to write an abstraction layer per se but rather by keeping our direct GL usage as isolated as possible, so that we can easily rewrite that stuff from scratch to really make good use of Vulkan, as you say. (This also allows us to support D3D if we want to.) Most of the logic here—batching, culling, texture atlas management, display list optimization—is very general logic that will apply no matter what hardware API we end up using.
This is already the approach taken by Chrome (trace Chrome and you'll just see a bunch of full window resolution PutImage calls), and it's very slow over the network, I get around 1 fps.
Firefox on the other hand uses more X11 calls, and it's much more responsive, close to running it locally.
Actually, I believe NX already does this in the latest (proprietary) version, though I don't use it.
But video codecs does not give the same experience as running apps that are actually written to take some care about what/how it updates. Not least because the artefacts when they can't keep up are less consistent with what I at least expect from a desktop environment.
Just as mentioned in the sibling post, X11 isn't up the task for modern graphics.
EDIT: lots of typos
I guess this is referring to XRender. XRender might well be the right approach in theory. Unfortunately, XRender is pretty much a failed experiment as far as browsers are concerned. There was a lot of noise in the Linux community demanding its use, which is why it went into Firefox. But ultimately what people want is fast vector graphics in practice, which you can only get by tightly controlling the vector graphics stack. The reality on the ground is that you can't trust the X server to do vector graphics efficiently. The X11-remoting use case is a tiny, tiny niche that unfortunately simply isn't worth optimizing for, especially with Wayland around the corner.
All things that are described in the page seem pretty "normal" and "obvious" things to do, so it's not quite clear which things are not being done in current browsers.
http://on-demand.gputechconf.com/gtc/2012/presentations/S002...
At first I thought it might be a good idea to implement a gallium statetracker for such a framework but with the direct3d statetracker not being supported by projects like wine I thought maybe vulkan will open up an opportunity to implement something with similar performance which is driver,vendor and platform agnostic.
Note that one of the biggest benefits of WebRender is that it doesn't need to do any of that: if you render CSS directly, you can avoid all of these complex path rendering techniques.