Show HN: Dropflow, a CSS layout engine for node or <canvas> (github.com)
Live demo: https://chearon.github.io/dropflow/
It matches browsers in all cases I can find where they agree, and it's fast. It supports `position`, `inline-block`, `z-index`, and complex properties like `float` and `vertical-align`. It doesn't support high-level layout like flexbox or grid yet, but you can get intrinsics to easily divide space yourself and paint multiple layout trees. It has a great text layout implementation, and supporting non-Latin scripts is a top priority.
I've wanted this to exist as far back as 2013, and the desire kept coming up: for a way to get detailed intrinsics, for high quality rich text layout to canvas and SVG, and for server-side rich text. We currently use it in CellEngine for our new canvas-based spreadsheet library to layout text in hundreds of thousands of cells, and will be using it soon to render PDFs with thousands of pages in a few seconds.
145 comments
[ 3.5 ms ] story [ 211 ms ] threadresponsive layouts, removing the need to focus on multiple properties: https://flexboxcss.com
https://github.com/CRED-CLUB/synth-android
I wonder how hard it was to implement css. I've heard it can be pretty complex.
There's another project called Sciter that uses CSS to target native graphics libraries: https://sciter.com
> I wonder how hard it was to implement css. I've heard it can be pretty complex.
It was hard, but the biggest barrier is the obscurity of the knowledge.
Text layout is the hardest, because working with glyphs and iterating them in reverse for RTL is brain-breaking. And line wrapping gets really complicated. It's also the most obscure because nobody has written down everything you need to know in one place. After I finished block layout early on, I had to stop for a couple of years (only working a few hours a week though) and learn all of the ins, outs, dos, and don'ts around shaping and itemizing text. A lot of that I learned by reading Pango's [1] source code, and a lot I pieced together from Google searches.
But other than that, the W3C specifications cover almost everything. The CSS2 standard [2] is one of the most beautiful things I've ever read. It's internally consistent, concise, and obviously the result of years of deliberation, trial and error. (CSS3 is great, but CSS2 is the bedrock for everything).
[1] https://gitlab.gnome.org/GNOME/pango/
[2] https://www.w3.org/TR/CSS22/
Every time someone releases a new rich text editor I'm disappointed to find that it uses contentEdtiable. Would be very interesting!
Sciter is using its own implementation (obviously).
contentEdtiable thing is indeed quite limited for general purpose WYSIWYG editor.
For example Web platform is missing transactional update [1] mechanism that allows to put custom DOM mutation groups into unified undo/redo stack.
Sciter's <htmlarea> element ( implement behavior:richtext - WYSIWYG ) allows to build specialized editors. For example it is used in Sciter.Notes [2].
[1] https://docs.sciter.com/docs/behaviors/behavior-richtext#ric... [2] https://notes.sciter.com/
Your work might have been, or maybe still is, the worlds biggest chance for this to change!
Blink source is de facto current spec. Each function, if not single line, there is a paragraph in spec.
I remember at WHATWG / HTML5 WG times when Ian Hickson (Google) was pushing whole SQLite (and its SQL flavour) to be included in HTML5 ...
The spec area is so huge and indeed obscure that even Microsoft could not handle it.
It's not that MS couldn't handle its own engine, it was just not worth it for them in the long term.
Ladybird, NetSurf or Servo engines are prime examples that it is possible to create an independent web engine from scratch even not being a big corp. If they can do it, MS definitely could do it as well if they wanted.
I can confirm this. I've been working on a (much simpler!) text layout engine for my canvas library over the past couple of months and the amount of complexity associated with just stamping some glyphs onto a canvas has left me screaming at my laptop on an almost daily basis. Getting a decent underline was a proud moment!
Question: did you ever find out what algorithm the various browsers are using to calculate how many words can fit on a given line? I'm almost there, except words will occasionally jump between lines when I scale the text. Really annoying!
The PR's still a work in progress, but I've got all the functionality I want in there (shaping lines to fit in non-rectangular containers, styling text, text along a non-straight line, dynamic updates, etc). Just need to test and document it all now ... https://github.com/KaliedaRik/Scrawl-canvas/pull/75
This made me laugh because I can relate so much. Inline backgrounds can start in LTR text and end in RTL text, and when I was implementing that I got so frustrated that I had to stop and seriously consider I might have an anger problem.
> Question: did you ever find out what algorithm the various browsers are using to calculate how many words can fit on a given line?
Not sure if I understood the question correctly, but they use a greedy algorithm where the break points in the string are the choices. If your glyphs are scaling and so is the available width, you might have a float precision problem? Browsers use integers for that reason. I'm still using floats.
Scrawl/that PR look extremely cool! I would love to some day support CSS `shape-outside`, which achieves a similar thing to what you have in your PR.
i can understand the abuse for server side rendering, but at this poit you pretty much have a bad browser engine. running inside a 2d hack in a browser engine. sigh.
you can still follow css spec if you want. just get a better environment than a canvas.
But the idea of using html for text styling has stuck, Swing UI do text styling with html (and rudimentary css!) to this day. Html would not be a lingua franca if its use was limited to the equivalent of native speakers (browsers).
What I'm thinking is: there's been soo much discussion around "we have corrupted the Web", "Web standards were never made to build apps on", etc. with sometimes good and sometimes really bad arguments.
If we can build viewports in canvas that behave more like just a desktop environment where apps can be built, maybe that could be good? It could mean a split between the informational web and the "apps on the browser" paradigm, and in a ideal scenario this could make things simpler and more organized for everyone? Or it could just mean more work, more standards, more rupture and more siloing. Honestly don't know.
(It's also terrible for accessibility, for both disabled people or just regular users who expect to be able to navigate with the keyboard for instance, unless the framework re-implement everything itself, which I doubt)
In which case, we're able to make an "inspect element" tool, which we could use to copy text out, and at the very least draw black rectangles on top of adverts. https://xkcd.com/2677/
To get an idea of what you'd have to emulate, here are some (not even all) issues related to rendering text: https://faultlore.com/blah/text-hates-you/
Many of us just want good, GUI apps for many things. They were often faster and lighter. VS Code is the exception to the rule which I use regularly.
“…instead of sending a command message to the object on the server, the client would send the X-Y coordinates of your mouse click. The server would then render its own copy of the scene into an internal buffer to figure out what object you had clicked on. Not only was this extremely inefficient, but the race conditions inherent a multi-user environment meant that it also sometimes just got the wrong answer. It was amazing…”
http://habitatchronicles.com/2004/04/you-cant-tell-people-an...
Seems like less of a risk than the flash days because browsers are faster and more capable, and stuff like this is currently a subset of what the browser can do, instead of a superset like flash was (or the strong vendor push like microsoft silverlight).
Not old enough to have been a dev when flash was in it's prime though so might not have the most accurate view.
The default way of generating beautiful PDFs in the backend these days is, running a headless browser and using browser APIs to turn HTML/CSS into PDFs. And apparently, it's a bit costly running instances of browser in the server and scale it properly for huge workloads.
This is literally a game changer. Now it's possible to design PDFs using HTML/css and generate them without the browser overhead!
This impressive effort looks far better than what I'd achieve, but when this approach has been tried before, it is eventually discovered that few organizations have the resources to maintain a rendering engine long-term.
> few organizations have the resources to maintain a rendering engine long-term
I'm curious are there other instances of this happening than Edge switching to Blink? That event was one of my main motivators; it felt like further consolidation of obscure knowledge.
Very fun project! Did you ever consider integrating with web-platform-tests? It's shared between all the common browser vendors, and we're always interested in more contributors :-)
True. But I wonder if there are more special-purpose engines similar to Prince that have been abandoned.
> Did you ever consider integrating with web-platform-tests?
I've run some of the WPT tests manually, but I don't yet have <style> support, and some of them use <script> I think? That's a path I'm wary of (eval()?) but I could have a special mode just for tests.
I did discover lots of weird corners that would be great to make some WPT tests for. Definitely something I want to do!
Do you think the WPT would accept pull requests replacing such tests with equivalent tests that don't use <script> (perhaps using a build script to generate multiple tests instead - or simply writing out the tests longhand)?
I could run against only the ref-tests, but if I can't get full coverage then the WPT seems to provide little value over our own test suite.
[0]: https://github.com/DioxusLabs/taffy
As for the second option, I don't actually know. If it becomes 10x as long, I would assume you get a no for maintainability and readability reasons. If it's 20% longer and becomes no less clear, I'd say give it a try with a couple tests first? It's possible that the WPT would be interested in expanding its scope to a wider web platform than just browsers. You would probably never get people to stop writing JS-dependent tests, though, so you would need to effectively maintain this yourself.
Of course, for a bunch of tests you really can't do without <script>, given that a lot of functionality is either _about_ scripting support (e.g. CSSOM), intimately tied to it (e.g. invalidation) or will be tested only rather indirectly by other forms of tests (e.g. computed values, as opposed to used values or specified values or actual values or …).
A build script used by significant swaths of the test suite is almost certainly out; it turns out people like being able to edit the tests they're actually running. (We _do_ have some build scripts — but they're mostly just mechanically generating lots of similar tests.
A lot of the goal of WPT (and the HTML Test Suite, which it effectively grew out of) has been to have a test suite that browsers are actually running in CI: historically, most standards test suites haven't been particularly amenable to automation (often a lot of, or exclusively, manual tests, little concern for flakiness, etc.), and with a lot of policy choices that effectively made browser vendors choose to write tests for themselves and not add new tests to the shared test suite: if you make it notably harder to write tests for the shared test suite, most engineers at a given vendor are simply going to not bother.
As such, there's a lot of hesitancy towards anything that regresses the developer experience for browser engineers (and realistically, browser engineers, by virtue of sheer number, are the ones who are writing the most tests for web technologies).
That said, there are probably ways we could make things better: a decent number of tests for things like Grid use check-layout-th.js (e.g., https://github.com/web-platform-tests/wpt/blob/f763dd7d7b7ed...).
One could definitely imagine a world in which these are a test type of their own, and the test logic (in check-layout-th.js) can be rewritten in a custom test harness to do the same comparisons in an implementation without any JS support.
The other challenge for things like Taffy only targeting flexbox and grid is we're unlikely to add any easy way to distinguish tests which are testing interactions with other layout features (`position: absolute` comes to mind!).
My suggestion would probably be to start with an issue at https://github.com/web-platform-tests/rfcs/issues, describing the rough constraints, and potentially with one or two possible solutions.
We are now using a commercial alternative that seems be be using a custom engine that implements the HTML and CSS specs. The result is reduced memory usage (below 512MB during my tests) and the resulting PDF is much smaller, 3.3MB vs 42MB.
https://ekoopmans.github.io/html2pdf.js/
(Admittedly, the PDFs I generate are most certainly not beautiful, so maybe that's the difference)
But I wondered whether using instead something like LaTeX wouldn't be faster and easier to scale.
I think most folks probably don't realize how difficult it is to go from HTML -> PNG programmatically. You get hit with a thousand papercuts related to either Node<>Browser differences or HTML<>Canvas differences.
Satori is meant for rendering Open Graph images (e.g. the little images that come up when you post a link on Twitter/Slack/Facebook), but I found that it works well for rendering arbitrary images. Satori has no native dependencies, so it kinda "just works" on the backend. It supports a subset of modern CSS, including flexbox.
My use case is posting match reports for League of Legends into a Discord text channel, e.g. person X just played a match, here are their stats.
It's quite nice because there are almost zero server-side native dependencies (the one exception is the library to convert svg -> png requires some native libraries).
Here's what a match report looks like: [1]
Here's an example of what the JSX looks like: [2]
I also built a small project [3] that renders the JSX in a browser to make developing the images just as easy as developing a normal website.
If others are interested in this, I would by happy to write a blog post about the process.
[0]: https://github.com/vercel/satori
[1]: https://github.com/shepherdjerred/glitter/blob/main/assets/p...
[2]: https://github.com/shepherdjerred/glitter/blob/main/packages...
[3]: https://github.com/shepherdjerred/glitter/tree/main/packages...
Text is about the most complex UI element that a UI framework offers, and if you’ve got Flow Layout working, that’s very encouraging! Will be interested how far along things like gestures (particularly text highlighting) and IME integration are. In any case, kudos for opensourcing this!
[0] https://pagedjs.org/about/
I was looking at Glide Grid the other day, and it renders so fast, even with 1 million rows; it's somehow responsive. There should be an easier way to render HTML to canvas without resorting to low-level primitives. Why is canvas faster than the "regular" DOM renderer?
I'm only speculating, but it doesn't seem surprising that regular DOM rendering logic - which has to handle approximately a bazillion different rules and special cases - is slower than a custom renderer written for a specific subset of HTML.
You can have a fast DOM without canvas, but it requires creative thinking. DataGridXL also renders millions of cells, but it does not use canvas as its main renderer (https://www.datagridxl.com/demos/one-million-cells).
The way it works: only columns are their own DOM nodes. For browsers it's just too much to ask to re-render let's say (20rows*10cols) 200 DOM nodes while keeping scrolling at 60fps.
I don't think this is true with modern browsers and CSS. For a table, every cell and parents of the cells as much as possible, should be styled `contain: strict` and if possible, absolutely positioned.
Many grid components were developed many years before modern compositors and the `contain` property.
Browsers can handle AND update really large DOMs, but they still choke on doing all of these actions (repaint, reflow) WHILE SCROLLING, which is a different game.
You can actually insert your HTML into a SVG foreignObject, and then drawImage() that onto your canvas. But your HTML-in-SVG document will need to load all its own resources, fonts, CSS etc.. which makes this process rather tedious.
A web search shows something that's written in Rust: https://github.com/DioxusLabs/taffy?tab=readme-ov-file#taffy