Ask HN: I'd like to practice coding GUI from scratch. Any recommendations?
As the title says, I want to practice coding a GUI framework, probably in C++.
I read about a retained and an immediate modes, have a rough understanding how event loop works, and overall my goal is to practice architecture and optimization (especially cache-friendliness), and less so graphics and typography rendering, though I understand it's unavoidable to implement a rendering pipeline (I plan to start off with AGG or Skia graphics libraries).
The dummy app itself will be less about forms but more about data representation, e.g. an audio editor or a node-based system, where data should be updated and visualized in real time, and everything should feel responsive.
Could you please give some research directions? Maybe case studies, best practices, some interesting software, maybe to read more in detail about related architectures, etc.? Or maybe personal stories? I'm struggling to find related info which would not be focused on some framework, etc.
93 comments
[ 4.0 ms ] story [ 150 ms ] threadThat being said, I'm interested to see the range of answers to OP's question, partially as I'm curious what HN considers "from scratch" and partially because I've thought about doing the same thing.
Edit: Same thing making-gui-from-scratch, not the audio stuff.
This is the main reason I try to do this when I can. When building things for fun I try to do it from the lowest level of abstraction that I can so that I understand more of what the higher levels of abstraction cover up.
I guess I just like turning unknown unknowns into known unknowns.
But seriously from scratch in this case is a canvas plus possibly some primitives for marking the canvas. Cairo provides these things.
I used to have an old 2MB DOS notebook that I would fire up now and then to play with my ideas in this space. I could get mouse input through an INT op code of some sort, and there was the keyboard and getting access to video RAM.
I have tried to play with the Linux kernel drivers underlying things like wayland and X but I lack the determination to see that through.
Another post here someone listed the object model paradigms and that’s probably closer to what OP is interested in.
You need an event system also. Cairo does not provide this.
We wrote out own custom 2D canvas with Cairo for Ardour, but the event handling has to originate somewhere else (in our case, GDK).
What's particularly fun is that an optimally-compiled kernel can boot to userspace disorientatingly quickly - like, ~500ms on decade+-old hardware. So you can be running your code almost as the user's finger is still moving away from the power button (yeah, okay, notwithstanding BIOS/EFI dwell time).
QEMU is also just as fast (and has no POST delays), and rebuilding small initramfs images takes like a few hundred milliseconds if done carefully, so you can end up with iteration processes where you can tweak a line of code, hit ^S and have an updated initramfs running the newly-compiled code in a fresh boot in like 1 second or so. This was on a low-end i3 NUC.
As for what I did get working... uhh... I did get graphics on the screen in the end, but the actual hardware I was testing with proved a tad flaky at the time and I never had the first clue how to figure out what was going on, so I ended up moving on to other things. I did make reasonable progress on the QEMU yak-shaving front before getting to that stage though.
- Compartmentalizing yak-shaving QEMU distinctly from everything else might be accelerated slightly by borrowing your distro kernel for a bit (because they generally always have the modules necessary to reach an initramfs compiled-in).
- You can skip out on needing a disk image, partition scheme and conventional bootloader by passing `-kernel` and `-initrd` (and `-append` to set the boot arguments). This neatly solves an entire dimension of issues at once, and is awesome.
- Specifying `-serial mon:stdio` (and `-append 'console=ttyS0,115200'`) writes serial port output to QEMU's stdout *while* giving you a way to fall-through to the QEMU monitor using ^A c, and insta-kill the VM with ^A k (orrr it might be ^A q or ^A x or something, I don't remember lol). This is awesome for bringup and furthermore irreplaceably practical as a way to get guest-side stdout (aka printf() \o/) onto a consistently-located spot on the screen (ie no pesky windows opening in random locations etc). (As a point of comparison, the -curses option does a fancy thing where 80x25 VGA textmode is translated into a curses window - yup, booting MS-DOS this way is fun - but in this mode terminal scrollback doesn't work; -serial mon:stdio doesn't initialize curses, so terminal scrollback works normally/correctly.) All this is complimented well by `-nographic` which disables VGA output, likely markedly irrelevant beyond bringup, but probably quite useful to begin with.
- QMP, the QEMU Monitor Protocol, is a JSON-based just-verbose-enough-to-be-annoying control protocol that lets you do nice tricks like telling running QEMU instances to reset/reboot (fully restarting QEMU will definitely slow things down a bit). QMP can listen via TCP or a domain socket. Thankfully the initialization handshake is effectively one-way, so IIRC you can just stuff the "yes hello version 1 blah blah" down the wire along with the reboot request as a fixed payload crammed through netcat.
- "Oh but of course" surprising-but-not-surprising thing: in order for the text console to have any practical purpose, writes to it must be synchronous so that the "about to load X" has fully made it (to the VGA text region|to video memory|out the serial port) before X happens. That synchronicity is a truly significant source of boot-time slowdown, and (maybe second to jiggling what kernel code is compiled in, what's in modules, and what's left out), completely disabling boot messages is one of the secrets to kernels that load virtually instantaneously. This is straightforward - `-append 'loglevel=0'` - aaaaand naturally happens after bringup :). Once you get there (and have commandline reboots working), with all the noise gone (QEMU prints nothing itself) you can make decisions about things like whether you'd like to preserve the output from previous boots in the terminal scrollback (incidentally `tput clear` resets that).
- Playing with an initramfs-less kernel (or the distro stock initramfs) can be useful to figure things out, but gets boring after approximately 2 minutes. One frequently-used approach to generating initramfs images is `find | cpio --create --format=newc > initrd.img`, but the cpio method has the caveat...
I work on projects that mostly use buildroot to make their images, but I've also done that other ways when I was playing with an RPi.
My personal experience is that starting developing in opengl always ends up wasting hours getting the right libs installed to get potentially full 3d GPU acceleration - which might be beyond the point if you want to draw stuff.
Casey muratorri is spot on in saying that not having a single, simple, unniversal and basic API to "open a window and draw pixels" is what we lost the most during the 90s.
Anyway, try to bypass the not-interesting-to-you parts, and have fun !
I don't have any suggestions for how to build such a framework, but I would encourage you to play around with a few existing frameworks to see how they are designed, what you like about them, and what ideas to avoid.
For just one example, see the difference between managing the event loop entirely by hand with Win32[1] and using "signals and slots" in Qt[2]. There is a lot more to UI frameworks than this, and they vary on many more aspects.
- [1] https://docs.microsoft.com/en-us/windows/win32/winmsg/using-...
- [2] https://doc.qt.io/qt-5/signalsandslots.html
Something that's special with Qt is that signals and slots are declared in the class using custom syntax, e.g.
Yes here "signals:" and "slots:" look like the same kind of keyword as "public:" or "private:" in a class declaration, even though internally they're #defined as "public:" and "/**/" respectively.Qt makes use of a tool called MOC – the Meta Object Compiler – to generate code for signals and slots. It's not a pre-processor but a code generator, which has to run before you can actually compile the code. I think you could probably implement a similar system of event subscriptions without this additional code generation step.
See this page for details: https://woboq.com/blog/how-qt-signals-slots-work.html
Gtkmm, or rather its dependent library libsigc++, does not require a separate build step like MOC, it can handle signals and slots "natively" because it used a more modern version of C++. Qt required MOC when it was first written, and they never moved away from it, despite the language having moved on to allow the required functionality all by itself.
xlib is quite a pleasure to use though, and it probably doesn't qualify as a "framework" that you don't want to use.
Open source software is resilient to 'death' and 'abandonment' issues plaguing closed source software.
I'm writing this as someone who is using a window manager (Window Maker) whose original developers abandoned it for years, then someone else picked it up some years ago to continue development and i personally have contributed bug fixes and features to it years after both of those.
OSS doesn't die for as long as there is someone out there who is interested on it.
I love Dear Imgui: https://github.com/ocornut/imgui
It is the simplest thing in the world. If you are starting and are going to do 3D graphics anyway you don't need state(you just redraw the screen 60 times per second).
Anything else is extremely sophisticated. I also loved Qt, but the policies got a little cumbersome, and went native.
The big problem is that with state there is a lot of complexity involved that is dependent on a platform, and once you pick one it is hard to change.
The big advantage of 3d graphics and dear imgui or any other open source software is that it works anywhere and you are not as dependent on a single company.
- Data update are the best part -- there's no synchronization needed with an IMGUI approach because the "model" and "view" are the same thing.
- adjust the event loop so it only redraws on changes (e.g. mousemovement) rather than every frame (unless you're drawing every frame anyways for a game)
- IMGUIs can have trouble with very large lists -- imagine you have a tree view with 20k objects, you're going to process each one of them each time through the gui, even though most of them are outside the visible list. But it's hard to only draw the "visible" ones since you don't know the future (e.g. imagine iterating a list where some items are hidden). Usually this isn't a big deal to work around, just keep it in mind.
- Biggest problem i've run into with IMGUI style is layout. Since you are processing widgets as you go, you can't adjust to future things. For simple layouts like stacked property sheets this is fine, but once you get beyond that it can be complicated. I'm playing with some ideas now where I use a constraint based layout to come up with "reference boxes" up front that then the IMGUI can use to layout but it's still a pretty open problem.
Re: the layout problem, it seems to be a factor in a lot of real-world apps(buildings, hardware design, etc.) and to the extent that it's "solved" by retained mode, it's a solution based on moving around the processing order so that you deal with a different set of constraint edge cases manually. As such I think it really is just a unsolved class of issues, which wasn't tackled before because it involved a degree of algorithmic complexity and resource use that wasn't on the table in the 1980's when the GUI was first widely adopted.
So I think it's fine to explore having a whole algorithmic path and data structures dedicated to building the bounding targets for both collision and rendering - I have done similar when I've poked around at doing my own framework. That reflects the actual complexity of the solution, and it becomes especially apparent how deep you could go once you allow your targets to be arbitrary shapes with 2D transforms.
Not even close...
https://github.com/mitsuba-renderer/nanogui
https://github.com/rxi/microui
https://www.fltk.org/
https://github.com/achimdoebler/UGUI
I thought about writing a cross platform GUI toolkit, that would start from being a cross platform accessibility library.
Installing a screen reader like NVDA and observing how it interacts with the screen might be interesting to learn about the scope/depth of the accessibility APIs in general. (And then there's observing how screen readers handle different websites...)
Imgui and similar generally have zero accessibility support whatsoever, although it's certainly not impossible to add - just a bunch of messages back and forth to the accessibility layers in the OS - but the support would wind up being a module, and a bolted-on one at that that would have very little uptake because everyone who didn't need it (for themselves) would just compile it out. Whereas with Qt and Electron it's kind of like rabbithoooooookaybackawayfromtheinternals and everyone just leaves all the bits in :P and the support just comes along for free. (An interesting study in the tradeoffs between minimalism, modularity, and the commoditization of support of niche use cases...)
Also, regarding practice of UI architecture and optimization, I actually sometimes think about the idea of drive-by contributing to open source projects - they have pre-existing contexts and conventions that must be adjusted to (an important skill that is tricky to develop alone), and it's possible to approach them with an objective impartiality that is sometimes tricky to square with emotional investment in personal projects. This may be one approach to independently evolving mechanical abilities (structure, domain-specific mental modeling and problem-solving, best practices) separately from fundamental personal development, which may (seems to?) go a little slower.
GUI frameworks/toolkits/libraries require drawing but also require an event handling framework, which Skia does not provide.
https://www.cs.cmu.edu/~bam/uicourse/05631fall2021/
The second is Dan Olsen's book Developing User Interfaces, which has all of the details of how GUIs work, from graphics to interactor trees to events to dispatching. For some reason, it's absurdly expensive on Amazon right now.
Both Dan Olsen and Brad Myers were early pioneers in GUIs and GUI tools, so you'd be learning from the masters.
The videos are locked behind a login. Does one need to be a student?
Amazon bots have noticed that your comment is on HN so lot of people might buy it, that is why they jacked up the price :)
Would you be interested in a 'no restrictions, informal' collaboration?
I have little to no experience writing GUIs, but your "dummy apps" sound to me like the sorts of things I could populate the 'guts' of, to make useful.
For instance, I work a lot with audio signal processing, have considerable experience in that area, and am "half-assedly" (low priority side project) writing tools that would benefit from a GUI.
My email is on my profile.
The following design patterns are widely used with graphical user interfaces: observer-design pattern; model-view-controller; model-view-presentation; two-way-data binding; property binding; command design pattern; and composite design patterns for representing a collection of objects as single object.
For understanding event loop it may be much easier to implement a Xterm or VT100 keyboard-driven terminal user interface TUI since this does not requiring dealing with too many backends.
X11 can work on Windows (many Xservers available, since forever), MacOS (used to ship with one, but now you've got to install one), and you can run an Xwayland in Wayland. So it's as cross platform as you can get. Might not look great, but then cross platform doesn't usually look great anyway.
If you're going to do X, avoid Xlib, it adds restrictive abstractions on top of X protocol and really confuses things. XCB is much closer to just reasonable interfaces to the protocol. Ultimately, X11 is a distributed systems communication protocol which happens to have graphical output as a side effect; understanding the communications part first lets you get the most out of it.
Pick a platform and make the best application you can on that platform.
True. But there are several entirely adequate cross-platform toolkits. Pick a toolkit, not a platform.
(GTK+ (via gtkmm) user for 23 years, on 3+ platforms)
Most applications just pick Windows win32 API or MacOSX and forgets about everything else and are never released for Linux or other Unices, unless the application uses electron, Qt with C++, Qt with Python or Java SWING like JetBrains IDEA IDE family. The main problem of Linux desktop is the lack of a high level graphics library toolkit with a stable API, ABI and a C interface that does not introduce breaking changes on every release. Unlike Gtk, Qt is more stable, but it lacks a C API and C++ has a fragile ABI not friendly to foreign-function interface or cross-language linking.
Good UI frameworks are composable, and progressively expose their API users (developers) to their constituent parts, which can include systems such as drawing/rendering, animation, compositing, event handling, text editing, view hierarchy, navigation, accessibility, and so forth. As you build more complex applications with a UI framework, you naturally find yourself customizing default behaviors or implementing new controls or functionality on top of the existing capabilities. You start to see how the framework itself is built just by using it.
Once you reach expert level with a UI framework, you will have a conceptual understanding of how high level features of the framework are implemented in terms of the lower level functionality. As an expert, you will feel confident that you could implement a new type of control, or reimplement existing functionality such that your version is a perfect peer to the built-in functionality from the library author on all important axes (developer API, performance, user experience, etc). In some cases this might be a lot of work, but at least you should know generally how to go about it if you had to.
See if you can get to expert level with at least 2 different styles of UI frameworks. At that point, you will be capable of building your own.
But almost certainly should not do so.
> Good UI frameworks are composable, and progressively expose their API users (developers) to their constituent parts, which can include systems such as drawing/rendering, animation, compositing, event handling, text editing, view hierarchy, navigation, accessibility, and so forth.
I’m curious what would be good examples and bad examples for you here?
My first real UI kit was with VisualWorks Smalltalk many years ago. For me, this idea of composability and exposure was really strong here. It wasn’t always the greatest code, but all the source was exposed in the class library, and so you could just use the SelectionInList or you could dig into it, put breakpoints in it, rally take it apart and learn how it all worked at whatever level of abstraction you wanted to wade into.
UIKit/Cocoa didn’t provide that level of exposure because it’s a closed source binary, but there was a time when the documentation was pretty good. And much of that still persists today. And much of it was honed by many years of NextStep development, so there’s a certain consistency to much ( but not all) of it. So it hasn’t been as good, but it’s been decent.
Then there’s been Android. This has been the worst. Early to market. Continuously evolved. Chasing the latest trend in UIs. Historically sparse documentation, and when you do find stuff through searching, good luck figuring out the relevancy. So this has been the worst for me.
I’d be curious which of the toolkits you’ve advanced in have been strong in your rubric, and which less so?
I personally have learned the most from Cocoa, but systems like React and Dear ImGui have definitely shown me new ways to think recently.
In terms of contrasting what's strong vs weak, look at how Cocoa changed from AppKit to UIKit. They kept a lot of the same ideas: such as runloops, target/action, view hierarchy, and the responder chain, but did away with some things that were redundant (NSCell) or poorly suited to producing fluid UIs (timer based animations). Put another way, they improved composability (everything is a view as opposed to some things being views, others cells, and others still windows), and they introduced a new low level system in CoreAnimation that provided compositing and animation and made it a fundamental building block.
---
An unpopular opinion I hold is that a good API design is more important than source code. Like, pick any method on UIView: strong Cocoa programmers could write a workable implementation given just the method signature and the documentation if they had to. The design of the API and how it all fits together is the real magic. Once you get that, the implementation follows. ... on the other hand, Microsoft actually tried to do that once, and their implementation was bad: https://github.com/microsoft/WinObjC/tree/develop/Frameworks...
If you see yourself doing something 3+ times, automate it early!
Other than that, I do recommend the immediate mode GUI approach if you're writing everything yourself, as it's the lowest overall system complexity of all the approaches. It has a number of drawbacks, and things don't abstract/compose as easily as a standard object-oriented retained, or a newer React/SwiftUI approach, but exposing the raw parts has its upsides.
There is a start of a native Rust GUI which you could contribute to: https://github.com/linebender/druid
Best shade thrown on HN all week! Bravo.
Rust for instance has no overhead of garbage collection, while Erlang does garbage collection at the end of every function.
https://caseymuratori.com/blog_0001
https://handmadehero.org/
GUI frameworks are like cryptography: you never want to roll your own. You will spend endless time implementing all the widgets a typical UI framework has, tweaking it until it feels kinda right, and even then they won't feel totally right until you've done extensive user testing and acted on the resultant feedback. Hundreds or thousands of hours of work for something that won't be as good as Windows, macOS, or Qt.
And I haven't even gotten started on internationalization or accessibility! Ohohohoho, boy.
There's a reason why the best (only good, really) UI framework is proprietary to a large, multi-trillion-dollar company.
In conclusion, just use your OS's native framework or one of the half-decent ones for Linux if you're working on Linux.