> In Rust you can use ?, which is quicker than unwrap, but you also have to do something like Result<Type, Box<dyn std::error::Error>
No, you don't! When returning Result<T, E>, you can use ? on any Result<T2, E2> where E: From<E2>. In other words E doesn't have to be Box<dyn>, it just has to satisfy From<E2>.
Crates like thiserror make it easy to define your own error types that can be easily converted from all the other error types you can encounter in your functions! I do this whenever it doesn't make sense to use an inner error type directly, or whenever a function does its own validation that deserves its own error variant.
On top of that, I tend to also have error variants for each place in the function rather than only for each type of error, so at error sites I can do something like `.map_err(MyError::PlaceInTheFunction)?` rather than only `?`, and then errors become like a very efficient backtrace :)
Why does everyone love being able to control the allocator in Zig? I rarely want this low level control over the allocator, I would rather have an easy to use interface to malloc/free (such as RAII) or would prefer the language push me to avoiding allocations when unnecessary. C and Zig users are always talking about how arena allocators are easier to use than the alternative but I disagree because the alternative is extremely easy to use when the language makes it so.
Arenas have a lot of footguns like growing out of control if you don't have space in your logic to delete them (e.g. suppose every new HTTP connection required obtaining a new allocation from the primary arena -- now you have a memory leak). They also aren't necessarily faster than malloc: I remember hearing a case where mimalloc was faster than an arena for a one-shot program like a linker because reclaiming space improved cache locality. Plus it's difficult to implement things like growable vectors or hashmaps using an arena without wasting space.
I just don't see how "futz about with the allocator constantly" and "use an arena" is actually a good default.
mimalloc aggressively reclaims space and it benefits our usecase (in our Dip database built on rust).
This is for a long running process where consuming compute and memory should only occur on processing a request and must revert to a baseline memory consumption once the request is completed.
malloc with default allocator always gradually increases the memory consumption. This continues with enough request bursts on our testing until the process runs out of memory. mimalloc doesn't.
On mobile, we are stuck with the default allocator. It is what it is.
I first encountered this behaviour on Linux desktop with Qt QML apps as far back as 2015. For a long time, I thought it was Qt QML ListView's inefficiency and didn't think much of it.
But in recent years, the phenomenon was visible with Dip which was built on rust.
Dip does massive parallel fanout using green threads ('may' coroutines) with work being NUMA pinned (software hash) using a thread per core model.
A default request may span up to _chunk_count_ jobs (default is 5k) configurable per request.
Every request (x 5k jobs) from the client on a stress test sees a jump of ~10-20 MB of memory. Sometimes the jumps use large swaths of memory - say 100s of MB and stabilizes for a while only to gradually climb up again later. Rinse and repeat - OOM kicks in and the process crashes.
Many times we thought we were leaking memory somewhere. But armed with the Qt experience a decade ago, we intuited maybe its the allocator - and switched to mimalloc on desktop executable.
Dip's Memory using mimalloc stabilizes at ~60 MB post request completion no matter how many requests you throw at it.
Pity that on mobiles we have to live with the default allocator.
If you don't like arenas then that will push you towards avoiding allocations :)
I don't love malloc / free, managing frees one-to-one with allocations is a nightmare. An HTTP server is actually a great example of a good fit for arenas... you can have an arena created per connection, and then free the arena when the connection closes. Now all of your allocations during that connection are guaranteed to be freed instead of having to keep track of them individually. Need something to live across connections? Use the general purpose allocator for those particular things.
For one shot programs, why bother freeing anything at all? The gpa or malloc or whatever else is fine there.
Personally I mostly lean on fixed-size buffers, but if I'm writing Zig it's just little toy programs where blowing up when the buffer runs out of room is fine. But for big non-toy programs it might be good to use fixed-size buffers anyway and define your limits in terms of things actually relevant to your domain.
RAII and smart pointers are fine too, but C++ has you covered if you want those. Zig is specifically avoiding that kind of invisible code that destructors bring you.
> If you don't like arenas then that will push you towards avoiding allocations
I mean push in the sense of a pit-of-success, not making me do things because the alternative is unpalatable.
> managing frees one-to-one with allocations is a nightmare
It really isn't a nightmare if your language has tools to help you. I'm never particularly concerned about frees in Rust outside of thinking about room to optimize.
> Need something to live across connections? Use the general purpose allocator for those particular things.
Well, this is a determination you have to make very carefully and if you screw it up (like in the situation I was trying to outline) you have a subtle bug.
> Personally I mostly lean on fixed-size buffers, but if I'm writing Zig it's just little toy programs where blowing up when the buffer runs out of room is fine.
What's also fine is using malloc, which will have perfectly reasonable performance for your toy programs.
This is reinforcing my belief that Zig programmers actually just have a sort of aesthetic attachment to the idea of managing their allocations themselves and this has very little to do with the actual practical concerns of writing software that works. I'm a lot more sympathetic to the other reply that talked about library code improving flexibility for those relying on the library, than this comment that only thinks about the case where everything goes perfectly and using a fixed size buffer for no reason.
> I'm never particularly concerned about frees in Rust outside of thinking about room to optimize.
Zig's position, love it or hate it, is that you should be thinking about it. Dropping a Vec<String> in Rust is O(n). Have you ever clicked the X on an app and it takes 10 seconds to disappear? Stuff like this is part of the problem.
If you have to write that for loop, the O(n) will jump out at you and push you towards an arena or FixedBufferAllocator or std.process.cleanExit, or any other solution to avoid the loop.
Dropping vecs that contain strings does not make any Rust code in the real world slow. You would need to have a vec containing billions of strings, occupying several gigs of memory, before dropping such a structure took any perceptible amount of time. And for extreme cases like that, you can simply use arenas. The bumpalo crate is approximately as old as Zig is. The reason that people so rarely turn to arenas in Rust isn't because you can't do them, because you've always been able to, but rather because arenas are a very specific solution to a specific sort of memory access pattern, and going out of your way to strictly adhere to that pattern doesn't produce measurable differences for most software.
Disclaimer at the start: Rust is my favorite language, this very much isn't an attack on Rust.
> Dropping vecs that contain strings does not make any Rust code in the real world slow. You would need to have a vec containing billions of strings, occupying several gigs of memory
I actually had a hobby project where that became very noticeable/too slow for my requirements at around 800k Strings, so several orders of magnitude less than your estimate. Switching to a solution that freed all the memory at once predictably made it run effectively instantly.
> And for extreme cases like that, you can simply use arenas
I think this is a fundamentally flawed way of thinking about things. Yes, each individual string allocation and drop isn't going to cost much, but in combinations with all the other similar decisions you're likely making if you think this way, plus the decisions of library authors, etc, it can add up even if strings specifically aren't a major problem. Avoiding pessimization by default instead of only when you can't ignore them any more is a good habit to be in.
Arenas aren't a free lunch either. You need a specific sort of memory access pattern, where large, conceptually related chunks of memory are being rapidly and predictably reclaimed, and if you don't have such a pattern then arenas are ceremony for no benefit. And the performance tradeoff is that you end up using more total memory than RAII because you're no longer eagerly reclaiming memory the moment it stops being used, but purposefully delaying your frees so that you can batch-reclaim it all at once. There are many cases where this makes sense, which, to reiterate, is why crates like bumpalo exist, which work gloriously well with Rust's lifetime system.
Just to let you know, `std::process::exit(...)` in Rust does not call any destructors. C++ also has `std::quick_exit(...)`. Applications exiting slowly is a problem caused by app developers.
(There are cases where it also takes time because the OS does have to flush all open files during process teardown, etc., but this shouldn't generally be user-visible.)
So often when people insist that a thing they don't like is "part of the problem" they haven't even thought about how much that "part" would be, although I suppose it's worse when they have thought and decided to go ahead anyway because the weasel wording might convince the hard of thinking.
A guy who rounded up from $14.86 to $15 on some grant paperwork and got 14 cents he wasn't entitled to out of a government programme, and Ken Paxton inexplicably becoming very wealthy from a government job are both "part of the problem" but hopefully it's apparent which of these is worth fixing.
> This is reinforcing my belief that Zig programmers actually just have a sort of aesthetic attachment to the idea of managing their allocations themselves and this has very little to do with the actual practical concerns of writing software that works.
Indeed if you don't want to manage everything yourself I don't think there's a need to use Zig. The whole pitch is making everything explicit and not hiding control flow. And I do think that limits its usefulness to smaller-scale codebases.
Writing quality software doesn't require much from the programming language until your codebase gets very large and you're collaborating with a ton of people. If I'm expecting a codebase to grow to a large scale I'm not reaching for Zig for that. I don't trust _myself_ to keep that much Zig code tidy and correct let alone a big mob of developers and agents :)
With a small program, it's rarely difficult to work out what's going on. If your small program develops a performance problem, it's usually fairly tractable to work out what's going on, and modify it. In extremis, you can just rewrite the whole thing.
Large programs are another matter. Hidden control flows in C++ (for example) can make pinning down performance problems extremely difficult. And then widespread refactoring can be extremely challenging.
Zig forces everything to be an explicit interface, which should make maintenance work on large programs much more tractable.
> Now all of your allocations during that connection are guaranteed to be freed instead of having to keep track of them individually
And how do you size the arena?
> keep track of them individually
I think this is an exaggeration of the problem. This is what RAII semantics in C++ are for.
> avoiding that kind of invisible code
The invisible code is traded for defer statements right after initialization that are obvious and therefore mostly just noise. I get the point of not wanting invisible code, but obvious defers are not much of an improvement either.
Two options: you either say "eh 20MB per request ought to be big enough" and resize it later if you need to. Or you take time to figure out reasonable limits in terms of whatever domain you're dealing with, and return descriptive errors when those limits are exceeded. Maybe you're saying your service supports up to N HTTP headers and M query parameters in the URL and so on. For almost anything there's a limit you can place that's reasonable. Having no limits at all just means the limit is when you run out of memory / disk / whatever.
If you like RAII and C++ just use that though, Zig is an alternative not an upgrade, imo.
Yup! In C you can mmap out a big anonymous region and then it will only take up actual resident space for pages you use. I'm not sure if Zig exposes this in the stdlib somewhere or if you'd just do it through libc.
If you're doing something like per-request or per-frame arenas you can clear the arena without having to reallocate the whole mapping. And depending how fancy you want to get you can mark the pages unused when you clear the arena so that the OS can reclaim them if needed.
If you had to pick a default, an arena allocator would definitely be it. Although even better would be a fixed-size arena that can never grow, like Tigerbeetle does.
I don't think the footguns you're talking about are real by the way.
arena's are popular in part because they are a way of grouping allocations by a single lifetime, reducing the amount of use-after-free's or double-free's that occur (reducing, not eliminating them).
they make a lot less sense in languages where the compiler or runtime takes care of this for you.
Passing allocator pointers around has the advantage of allowing you to customize the allocator when you want, and to ignore it when you want (which is most of the time), i.e. just use the allocator you were given and pass it around like some piece of syntax or ctx in Go.
The trouble comes when you have data structures that manage memory, and you want to associate the allocation strategy with the particular instance of the data structure. Or, worse, you don't store the allocator pointer with the data and you require that all functions operating on the data structure happen to use compatible allocators.
At its simplest, though, passing allocators around is just removing an implicit global variable that is sometimes helpful to customize locally, but nearly impossible to customize locally if left as a global.
Allocators help solve the problem of "this code I want to use is almost good enough; how do I actually use it?"
I distinctly remember one of my first "real" jobs, somebody had written a C# library for GraphQL. It had some sort of bug or lack of functionality I had to fix. The author had fallen hook, line, and sinker for sealed classes and all that nonsense, so the shortest path to using the rest of their code was to fork the library and re-write the thousand offending lines and all references thereto. Luckily it wasn't a deep dependency.
Contrast that with similar issues in typical Python. The library likely accepts my favourite duck-typed solution, if it doesn't then I can likely work around it with an appropriate choice of dunder methods in the code I have interacting with the library, and if it still doesn't yield then I can just rewrite (monkey-patch) the offending code at runtime.
Not that either of those scenarios are objectively optimal, but they illustrate one of the principles people have in mind when they use an interface like allocators (not the only one, I'm sure other commentors will talk your ear off), that when push comes to shove you'd prefer to be able to use your existing code to solve your new problems. And that does empirically mostly work here; you can use nearly the entire Zig stdlib (especially when ignoring things like the posix namespace) and most of the rest of the ecosystem as well on embedded devices, in WASM, in bounded-memory applications, in generic application code, in fast-running CLI apps not even bothering to free, etc.
I agree with much of your discussion about arenas by the way. They're a moderately poor fit for the current interface in Zig, largely because the free functionality is questionable at best. IME, most uses of arenas actually know they need such a thing. Functions taking allocator arguments and expecting the allocator to be an arena should take an arena argument. When using arenas for lifetime management, something should probably own that arena rather than implicitly assuming whatever gets passed through the stack is correct. Passing arenas to arbitrary functions expecting general-purpose allocators is prone to leaks, and you'd be better off using the arena (or a buffer created therefrom) as a backing allocator for a data structure capable of actually freeing if you can't trust the implementation you're calling into (and if you can trust it, they should expose a better API allowing you to detect that trustability at the callsite).
They're orthogonal concepts. Zig hasn't done this, but you could absolutely combine injected allocators and automatic destructors in the same language if you wanted.
The bigger design point is who's responsible for knowing where the object lives. Zig has a balance of power more in favour of the caller than the callee, a point I tend to agree with since it's quite easy for a caller to know exactly what they want done and quite hard for a callee to know what all possible callers might want done. That principle is applied pretty liberally through the rest of the language.
That just means you haven't run into the problems yet that a global allocator brings with it. Other people have and welcome the better control over allocation provided by the Zig std library. The challenge is of course to balance convenience and flexibility. Also if you write code that prefers to not allocate in the first place (which should be the default), you're not affected anyway.
I also don't get it, programming since 1986, in quite a good number of programming languages, including various kinds of systems programming alternatives.
To me this kind of stuff always feels a bit cargo cult programming.
if you don't want to, you can just pick the general purpose allocator and use it everywhere.
however, in languages that don't allow it, you can pick and choose different allocators for different workflows. it may not be important to you, but it is important to many. and shoe horning support for multiple allocators is not easy for language
You are looking at this from the wrong perspective. If you are fine with automatic allocations and RAII like you have in C++ or Rust, or with a garbage collected language, there is pretty much no reason to even look at Zig.
Zig is a language to use when you do need the control, mostly because of performance reasons. And once you are in, you start thinking how to manage your allocations and arenas are the logical outcome, once you notice that your application has scopes that can be destroyed at once.
If you need an argument why arenas are better than RAII, then you likely don't have the performance needs to even consider using Zig, because outside of performance, arenas are not better.
It is like saying "why do C++/Rust people don't want a garbage collector?", or "why do Haskell people insist on function purity?". A GC in C++ and Rust would make things much easier, and a few impure functions would likely make Haskell more practical, and then, if Rust let go of all that "memory safety" cruft, it would be much easier to write one-off scripts and prototypes with it.
Languages have opinions, that's why there are several of them and they are popular in some applications and not others. And Zig opinion is to be explicit so RAII is out, and the language does push you into avoiding allocation when unnecessary, as a side effect of not giving you the "easy to use interface" you want.
This makes it particularly suitable for fields like embedded software where you may not have malloc/free at all, but may be less so in a highly dynamic environment, where you may be better served with a GC anyways.
To me, that's not RAII, it is static analysis, useful, but different.
To me, the name "Resource Acquisition Is Initialization" implies that doing one means the other one is done implicitly. Now, in the end of the day, the result is machine language, with proper discipline you can follow the RAII principles in any languages, including assembly, but I have always understood RAII as being it done implicitly.
Also we can argue that Zig supports a weak form of RAII through the use of "defer", there are some nuances here.
Suppose you are implementing an application server (backend).
You can limit the concurrent requests that can be handled at a time to 2000 and limit memory per request handler to 2MB.
So you allocate 2000 x 2MB at program start.
Then you crate an arena allocator with (2MB limit) when starting to handle a request.
It will just fail if a request uses more memory than 2MB. This means you will basically never have OOM issues or a request using unexpected amount of memory to crash the server.
You also have performance benefits like:
- Allocation operation is basically an integer increment. - Memory for a request is laid out linearly in memory and likely is pre-populated so you won't have page fault into the kernel. - It is very unlikely to have double free or other memory management mistakes.
You can also fuzz the system in several ways to make sure requests can really never exceed that amount of memory usage so you don't have surprises in runtime.
If a request by a user really exceeds that much memory usage, you can just fail that request and log it so it will be debugged and fixed later.
You also don't need data structures like SmallVec (rust version of the thing) that are hybrid stack/heap allocated.
There are many intricacies to this approach but it works much better on scale when you have a large program that handles a large amount of memory.
It is super difficult to debug and fix memory usage issues on global allocator only programs in my experience.
I tried to fix one such issue on a database-like program in rust and it was impossible.
Also this means you don't absolutely need ownership/lifetime safety.
1. When I pick a low-level language, that's precisely the kind of control I'm after (otherwise, why go for a low-level language?).
2. Heap allocation is a very common cause of serious performance problems in low-level programs because malloc/free overhead is so high. Some languages have a moving GC to reduce the high overhead of heap allocation, but low-level languages typically can't use that optimisation, so you have to resort to arenas (which offer a similar optimisation to that of moving collectors). This isn't always faster than malloc/free, but it usually is, and if you want malloc/free you can pick that.
Anyway, I wouldn't say that the default in Zig is to use an arena, rather it's to give you the control that's the whole purpose of low-level languages.
Of the general-purpose methods it does; that's is exactly what these collectors were invented for - not to carry all the heavy overhead of malloc/free. I guess you could argue over the frequency of situations where it yields a significant boost. It's also true that malloc/free have also improved, especially in the very large runtimes (like TCMalloc and jemalloc), but they can't do all the optimisations moving collectors can.
It’s great that they were created for this purpose, but I have a faster allocator in non-moving GC engine in C++. And I don't have to moving gigabytes of data.
Generational moving allocators don't move objects so much, the total amount of work they need to do is substantially less than malloc/free allocators -- that's the whole idea of these algorithms and the reason why they're used (that's not under dispute; it's just the mathematics of the algorithms [1]). Still, there might be some situations where allocators in C++ could be faster, but if that were the case for my purposes, I would have stuck with C++. We needed very good performance for highly concurrent workloads, and C++ just doesn't come close in many of those (maybe at 10x the budget).
[1]: More precisely, the work they need to do doesn't scale linearly with the allocation rate as is the case with non-moving approaches. Of course, arenas are a simple case of a moving collector, so they also display the same effect, but they can't always be used.
> The time to solve performance, and get the 1000x wins, is in the design phase, when you can't profile.
How does this make practical sense? I find it immensely easier to fix performance issues by profiling, benchmarking, finding hotspots. Things that weren't obviously performance bottlenecks during design become obvious when you profile.
Interesting write-up though. I haven't properly tried zig but I like reading these "initial impressions" style articles about it.
There's a different between fixing performance mistakes, which tend to be obvious in profiling as big hotspots, and designing for maximum performance, which requires that you start, really, with the specification of what your application is doing, then follow that through to architecture and design with a strong view to reducing unnecessary work and staying within the operations that can be done at high speed with current systems.
While I don't agree TigerStyle inherently actually helps you with this, design and architectural mistakes don't usually get solved in the way you're describing. You sometimes have to rethink how to structure your entire codebase. Sometimes you can evolve an architectural improvement but sometimes it requires a complete rewrite and there's no way to get outside of that path dependency.
Most undergrad CS students go through an algorithms course and understand a bubble sort is inherently slower than a quick sort, no amount of optimization will get you past the fundamental n^2 time complexity. You can generalize those same analysis principles to whole architectures and distributed systems. If you don’t recognize that the server interactions your solution requires boils down to an n^2 algorithm, you’re in for a bad time. Unlike a function a live, deployed system is typically harder to evolve. Sometimes constraints like backwards compatibility are such that you simply can’t fix it.
> Most undergrad CS students go through an algorithms course and understand a bubble sort is inherently slower than a quick sort
That sounds about right, a DS&A course which introduces Hoare's Quicksort, which is old enough to draw a pension, as if that's the state of the art.
Introspective sorting is more than a quarter of a century old and yet probably most people with a CS degree do not even know it exists. What a poor excuse for an engineering discipline this is.
Their big-O studies will lead them to conclude, as a Fortran textbook I've studied did almost forty years ago, that Heapsort is the right choice. Same big-O best case as Quicksort, easier to write but a better worst case. Of course the Fortran textbook does have the heart to mention that Heapsort is actually noticeably slower - but same big-O as if that's what your users will care about.
With a following wind and enough free time (or if they ask an LLM) they might come up with introspective sorting, putting them only a few decades behind the state of the art. Amazing.
Zig's reimplementation of all of the standard libraries even though it's possible to use the existing ones is both baffling to me and kills usage of the implementation in my eyes.
Redoing critical parts such as crypto and compression is crazy in this day and age, hell they even have issues and merge requests open for issues that'd be headliners if found in any major language
Not just that, but on Windows, they're bypassing the actually guaranteed stable Win32 API in favor of linking directly to the lower-level components. Completely destroying any forward compatibility!
It's the kind of change that makes me read Zig as not trying to be a serious language. Not that there's anything wrong with that, but the way Zig positions itself is at odds with the completely bizarre takes they do.
The idea/hope is perhaps that the Zig programs will be of better quality than other programs, which will make it hard for the other side (the antivirus) to justify its position in a reasonable way. And in the end the other side will be forced to change its processes.
But the trust as a service (TaaS) platform could decide tomorrow that you are not trustworthy because, I don't know, it matched a pattern in a piece of ransomware which is actually a pattern you can produce by allocating 3 pages of memory in one call. Should you as a language now put compile time guarantees that 3 pages cannot be allocated?
This is obviously a ridiculous example, but what I'm aiming to say is, if a piece of software is matching something as untrustworthy when it's not, you shouldn't chase it down to force the little guy who's maxing good will software to fix it because the big bully thinks they are evil and deserve to get quarantined
If you spend some time working on cross platform code you will quickly realize that that choice is actually the right one. The NT API is actually very close in abstraction to Unix and allows for much less ad hoc code to handle windows specifics than using Win32 API (which is an abomination).
The zig windows NT implementations are very well done and I actually use them as reference when dealing with windows IO.
It's a shame really that Microsoft wont officially bless ntdll because it's a much more sensible API generally speaking.
Like why the hell did they throw away the return value and make everyone do the insane BOOL ret = Win32Function(); if(!ret) { DWORD err=GetLastError(); } dance?
The article says some Zig advantages but it's worth noting that all these advantages are tradeoffs.
Zig makes handling error easy but makes retaining error information hard. The result is that Zig developers either write more code than other languages to store diagnostic information, or just simply discard diagnostic information. The second makes it much harder to debug because the useful error information is discarded.
About the TigerStyle, it's worth noting that TigerBeetle doesn't even support variable-sized string type. You can store fixed-sized data, such as a string that's at most 32 bytes. What if the order information contains string longer than 32 bytes? Then you need to store that string in antoher database. TigerBeetle is not a general database. Static memory allocation doesn't fit most applicaitons.
The fallable allocation is useful in embedded system and OS development etc., but useless in common application programming, because OOM results in crash rather than failed allocaiton.
I'm always a bit surprised when people cite TigerBeetle for anything (Zig excellence, coding style, etc). They've made some unique choices that work great for them, but it really doesn't seem like they can be universally applied.
I'd bet only a small % of people praising TigerStyle actually disallow all runtime allocation in their own programs.
TB is… extremely forthcoming about this. They’re very explicitly building a specific sort of distributed state machine for high volume transaction data.
If they were building something else they’d end up with a different design. And I suspect a lot of TigerStyle would still apply if that other thing we’re interested in performance or correctness.
I want your honest opinion on Zig, for dev or internal tools I always use Rust. Probably not because its the best choice for that project, but simply because its more popular. Same reason everyone uses React instead of things like Solid or Svelte that I find much better than React. Is it the same case with Zig as well?
>"The IO parameter is even more powerful, even if it is newer. It is well worth the breaking changes required to bring it about (even if I skipped the 0.15.0 -> 0.16.0 transition). In Zig, every function that interacts with the underlying operating system (with caveats) requires an "IO" parameter. This provides the context for input, output, and everything else.
Porting Zig to new operating systems is mostly just porting IO."
Nice!
(In Software Engineering, I think with this change, Zig nailed Separation Of Concerns: (https://en.wikipedia.org/wiki/Separation_of_concerns) aka modularity/modular design, at least with respect to handling the asymmetries of different OS's I/O APIs/syscalls - rather nicely!)
That's because in Zig that would be seen as a surprise side effect - the allocator is expected when the action is performed, not when it's "prepared", this way you know append() allocates. Otherwise it wouldn't be explicit which Array methods allocated.
The Odin way is useful, too. Depends on what you're building.
The whole argument that you always know when Zig allocates is just wrong, as this shows. They “fixed” the hidden allocation by making it explicit in the append function but now you can pass in the wrong allocator. Not great is it? It is just all so fragile and relying on convention rather than hard checks.
People sometimes pursue a bad idea to incredible lengths and then everything else is justifiable based on that initial, bad idea that just causes much more pain than it solves.
That's just a matter of wrapping the allocator if you feel fhat is better approach. Question is, should there always be an allocator around with arr or do you want to split a build phase from an immutable use phase.
ArrayLists used to store an allocator at init time and then use it for appends and whatnot afterwards. It was decided that this hid allocations and wasn't appropriate for the standard library.
It is of course trivial to define your own arraylist type that wraps an allocator on initialisation if you so wish.
We used to have a version of ArrayList that bundled an allocator in the stdlib, we removed it because our experience having written Zig code for a few years led us to reach for the allocator-less variant almost always.
One aspect your argument is missing is that if you are juggling many ArrayLists, having each bundle its own copy of the allocator (instead of storing it just once) means wasting 16 bytes per ArrayList. The allocator-less variant is simply more flexible in that regard.
Odin ends up pointlessly carrying around Allocator structures for each such container, each of which is two words (so e.g. 16 bytes on a modern system) because of how Odin works.
So e.g. that growable array of i64 you've asked for - just the data structure itself, not the array which was the whole point - needs 40 bytes on a modern machine because it has 8 bytes for a pointer to the array, 8 bytes for the size of that array, 8 bytes for how much was used, 8 bytes for a pointer to the allocator procedures and 8 bytes for a (sometimes unused) metadata structure for that allocator.
It's not a huge amount of waste, but what you get for it isn't worth it.
In Zig they're at least not paying for this with runtime space, they're paying for it with the programmer's need to make sure they talk about the same allocator everywhere.
But instead you bloat up every call site with an extra argument that needs to be passed, which takes up a few bytes of instructions + claiming a register for it which could have been used for other arguments (and having the allocator in a register is probably a waste since it's a hopefully rare thing to need to access).
Hypothetically the code is consolidating sites where allocation happens. Zig’s design seems to really push you toward this by making anything else a PITA. So then that allocator reference is being used frequently and already sitting in a register anyway.
The few extra instructions for explicitly passing it might be small compared to the cost of having objects carry their own references. That would grow each instance by ~8 bytes. It would mean that the pointer to the allocator needs to be pulled from a unique memory location for each instance, which is maybe an order of magnitude slower than already having it in a register. Larger data structures would also mean more cache misses, which are another order of magnitude slower.
And this is all chicken feed compared to actually taking a walk of shame out to the DRAM to fetch more memory. So the language ergonomics encouraging you to consolidate your allocation might trump all other considerations in practice.
Note that in Odin you are already paying for an extra register because you silently move a magic "context" variable around in a register.
I haven't looked at the details in Odin, but it's definitely sensible to have the growth decision inline, which means only the (rare) growth itself needs the extra parameter.
You can, but Odin does not mark these procedures contextless and indeed it mustn't because in some cases it will fill out the allocator values by looking in the context, which it couldn't do if it did not have context.
But you need to keep track of the allocator somewhere, as otherwise you don't know how to deallocate a container, or its contents. You either pay a very small cost in memory, or have to structure all your code to keep track of the allocator for everything you create.
Or it can just live in the type system, as it does in Rust. This is basically always going to be cheaper, and for the case where you just want a single global default allocator it's free instead of two pointers.
I think with Rust 1.100 next month you could write an OdinStyleAllocator type which impl std::alloc::Allocator and has the two pointers inside it, and do the thing Odin does here if you wanted that but I expect you'd snap out of it.
There's always a hard sell with Zig. Unpopular opinion but I'm sorry Go and Rust made big leaps in different areas, and Zig is just always going to be a faster horse.
>Unfortunately, it breaks outside of directories without a build.zig.zon
Are you telling me someone was building a tool to switch versions left and right and did not have the thought to name the build file build.zig.zag? Wasted potential
What about the completly useless ZLS with comptime? I find it to be really a dealbreaker...
ex: with Zalgebra. Vec3.new doesnt give any autocompletion or error... neiter when you assign it to a variable v = vec3.new(0,0,0); v. you get no autocompletion or variable type you have to read GenericVector and try to understand it... Its like coding in Notepad.
104 comments
[ 0.22 ms ] story [ 7.2 ms ] threadNo, you don't! When returning Result<T, E>, you can use ? on any Result<T2, E2> where E: From<E2>. In other words E doesn't have to be Box<dyn>, it just has to satisfy From<E2>.
Crates like thiserror make it easy to define your own error types that can be easily converted from all the other error types you can encounter in your functions! I do this whenever it doesn't make sense to use an inner error type directly, or whenever a function does its own validation that deserves its own error variant.
On top of that, I tend to also have error variants for each place in the function rather than only for each type of error, so at error sites I can do something like `.map_err(MyError::PlaceInTheFunction)?` rather than only `?`, and then errors become like a very efficient backtrace :)
In other words implementing that trait lets you set a return type like Result<T, CustomAppError> use ? on it.
Arenas have a lot of footguns like growing out of control if you don't have space in your logic to delete them (e.g. suppose every new HTTP connection required obtaining a new allocation from the primary arena -- now you have a memory leak). They also aren't necessarily faster than malloc: I remember hearing a case where mimalloc was faster than an arena for a one-shot program like a linker because reclaiming space improved cache locality. Plus it's difficult to implement things like growable vectors or hashmaps using an arena without wasting space.
I just don't see how "futz about with the allocator constantly" and "use an arena" is actually a good default.
This is for a long running process where consuming compute and memory should only occur on processing a request and must revert to a baseline memory consumption once the request is completed.
malloc with default allocator always gradually increases the memory consumption. This continues with enough request bursts on our testing until the process runs out of memory. mimalloc doesn't.
On mobile, we are stuck with the default allocator. It is what it is.
Curious, how much memory and how many requests does that take to exhaust?
But in recent years, the phenomenon was visible with Dip which was built on rust.
Dip does massive parallel fanout using green threads ('may' coroutines) with work being NUMA pinned (software hash) using a thread per core model.
A default request may span up to _chunk_count_ jobs (default is 5k) configurable per request.
Every request (x 5k jobs) from the client on a stress test sees a jump of ~10-20 MB of memory. Sometimes the jumps use large swaths of memory - say 100s of MB and stabilizes for a while only to gradually climb up again later. Rinse and repeat - OOM kicks in and the process crashes.
Many times we thought we were leaking memory somewhere. But armed with the Qt experience a decade ago, we intuited maybe its the allocator - and switched to mimalloc on desktop executable.
Dip's Memory using mimalloc stabilizes at ~60 MB post request completion no matter how many requests you throw at it.
Pity that on mobiles we have to live with the default allocator.
I don't love malloc / free, managing frees one-to-one with allocations is a nightmare. An HTTP server is actually a great example of a good fit for arenas... you can have an arena created per connection, and then free the arena when the connection closes. Now all of your allocations during that connection are guaranteed to be freed instead of having to keep track of them individually. Need something to live across connections? Use the general purpose allocator for those particular things.
For one shot programs, why bother freeing anything at all? The gpa or malloc or whatever else is fine there.
Personally I mostly lean on fixed-size buffers, but if I'm writing Zig it's just little toy programs where blowing up when the buffer runs out of room is fine. But for big non-toy programs it might be good to use fixed-size buffers anyway and define your limits in terms of things actually relevant to your domain.
RAII and smart pointers are fine too, but C++ has you covered if you want those. Zig is specifically avoiding that kind of invisible code that destructors bring you.
I mean push in the sense of a pit-of-success, not making me do things because the alternative is unpalatable.
> managing frees one-to-one with allocations is a nightmare
It really isn't a nightmare if your language has tools to help you. I'm never particularly concerned about frees in Rust outside of thinking about room to optimize.
> Need something to live across connections? Use the general purpose allocator for those particular things.
Well, this is a determination you have to make very carefully and if you screw it up (like in the situation I was trying to outline) you have a subtle bug.
> Personally I mostly lean on fixed-size buffers, but if I'm writing Zig it's just little toy programs where blowing up when the buffer runs out of room is fine.
What's also fine is using malloc, which will have perfectly reasonable performance for your toy programs.
This is reinforcing my belief that Zig programmers actually just have a sort of aesthetic attachment to the idea of managing their allocations themselves and this has very little to do with the actual practical concerns of writing software that works. I'm a lot more sympathetic to the other reply that talked about library code improving flexibility for those relying on the library, than this comment that only thinks about the case where everything goes perfectly and using a fixed size buffer for no reason.
Zig's position, love it or hate it, is that you should be thinking about it. Dropping a Vec<String> in Rust is O(n). Have you ever clicked the X on an app and it takes 10 seconds to disappear? Stuff like this is part of the problem.
If you have to write that for loop, the O(n) will jump out at you and push you towards an arena or FixedBufferAllocator or std.process.cleanExit, or any other solution to avoid the loop.
> Dropping vecs that contain strings does not make any Rust code in the real world slow. You would need to have a vec containing billions of strings, occupying several gigs of memory
I actually had a hobby project where that became very noticeable/too slow for my requirements at around 800k Strings, so several orders of magnitude less than your estimate. Switching to a solution that freed all the memory at once predictably made it run effectively instantly.
> And for extreme cases like that, you can simply use arenas
I think this is a fundamentally flawed way of thinking about things. Yes, each individual string allocation and drop isn't going to cost much, but in combinations with all the other similar decisions you're likely making if you think this way, plus the decisions of library authors, etc, it can add up even if strings specifically aren't a major problem. Avoiding pessimization by default instead of only when you can't ignore them any more is a good habit to be in.
(There are cases where it also takes time because the OS does have to flush all open files during process teardown, etc., but this shouldn't generally be user-visible.)
Measure, measure, measure.
So often when people insist that a thing they don't like is "part of the problem" they haven't even thought about how much that "part" would be, although I suppose it's worse when they have thought and decided to go ahead anyway because the weasel wording might convince the hard of thinking.
A guy who rounded up from $14.86 to $15 on some grant paperwork and got 14 cents he wasn't entitled to out of a government programme, and Ken Paxton inexplicably becoming very wealthy from a government job are both "part of the problem" but hopefully it's apparent which of these is worth fixing.
Indeed if you don't want to manage everything yourself I don't think there's a need to use Zig. The whole pitch is making everything explicit and not hiding control flow. And I do think that limits its usefulness to smaller-scale codebases.
Writing quality software doesn't require much from the programming language until your codebase gets very large and you're collaborating with a ton of people. If I'm expecting a codebase to grow to a large scale I'm not reaching for Zig for that. I don't trust _myself_ to keep that much Zig code tidy and correct let alone a big mob of developers and agents :)
With a small program, it's rarely difficult to work out what's going on. If your small program develops a performance problem, it's usually fairly tractable to work out what's going on, and modify it. In extremis, you can just rewrite the whole thing.
Large programs are another matter. Hidden control flows in C++ (for example) can make pinning down performance problems extremely difficult. And then widespread refactoring can be extremely challenging.
Zig forces everything to be an explicit interface, which should make maintenance work on large programs much more tractable.
And how do you size the arena?
> keep track of them individually
I think this is an exaggeration of the problem. This is what RAII semantics in C++ are for.
> avoiding that kind of invisible code
The invisible code is traded for defer statements right after initialization that are obvious and therefore mostly just noise. I get the point of not wanting invisible code, but obvious defers are not much of an improvement either.
Two options: you either say "eh 20MB per request ought to be big enough" and resize it later if you need to. Or you take time to figure out reasonable limits in terms of whatever domain you're dealing with, and return descriptive errors when those limits are exceeded. Maybe you're saying your service supports up to N HTTP headers and M query parameters in the URL and so on. For almost anything there's a limit you can place that's reasonable. Having no limits at all just means the limit is when you run out of memory / disk / whatever.
If you like RAII and C++ just use that though, Zig is an alternative not an upgrade, imo.
If you're doing something like per-request or per-frame arenas you can clear the arena without having to reallocate the whole mapping. And depending how fancy you want to get you can mark the pages unused when you clear the arena so that the OS can reclaim them if needed.
"Memory is cheap" and all that? :-)
All HTTP servers have a configurable maximum request size. Ex: client_max_body_size in Nginx.
...which typically has more overhead than a good GC though and much more overhead than using arenas for lifetime management.
Should you allow an http-server to allocate unbounded request sizes? Seems like it would lead to an easy denial of service.
I don't think the footguns you're talking about are real by the way.
they make a lot less sense in languages where the compiler or runtime takes care of this for you.
The trouble comes when you have data structures that manage memory, and you want to associate the allocation strategy with the particular instance of the data structure. Or, worse, you don't store the allocator pointer with the data and you require that all functions operating on the data structure happen to use compatible allocators.
At its simplest, though, passing allocators around is just removing an implicit global variable that is sometimes helpful to customize locally, but nearly impossible to customize locally if left as a global.
I distinctly remember one of my first "real" jobs, somebody had written a C# library for GraphQL. It had some sort of bug or lack of functionality I had to fix. The author had fallen hook, line, and sinker for sealed classes and all that nonsense, so the shortest path to using the rest of their code was to fork the library and re-write the thousand offending lines and all references thereto. Luckily it wasn't a deep dependency.
Contrast that with similar issues in typical Python. The library likely accepts my favourite duck-typed solution, if it doesn't then I can likely work around it with an appropriate choice of dunder methods in the code I have interacting with the library, and if it still doesn't yield then I can just rewrite (monkey-patch) the offending code at runtime.
Not that either of those scenarios are objectively optimal, but they illustrate one of the principles people have in mind when they use an interface like allocators (not the only one, I'm sure other commentors will talk your ear off), that when push comes to shove you'd prefer to be able to use your existing code to solve your new problems. And that does empirically mostly work here; you can use nearly the entire Zig stdlib (especially when ignoring things like the posix namespace) and most of the rest of the ecosystem as well on embedded devices, in WASM, in bounded-memory applications, in generic application code, in fast-running CLI apps not even bothering to free, etc.
I agree with much of your discussion about arenas by the way. They're a moderately poor fit for the current interface in Zig, largely because the free functionality is questionable at best. IME, most uses of arenas actually know they need such a thing. Functions taking allocator arguments and expecting the allocator to be an arena should take an arena argument. When using arenas for lifetime management, something should probably own that arena rather than implicitly assuming whatever gets passed through the stack is correct. Passing arenas to arbitrary functions expecting general-purpose allocators is prone to leaks, and you'd be better off using the arena (or a buffer created therefrom) as a backing allocator for a data structure capable of actually freeing if you can't trust the implementation you're calling into (and if you can trust it, they should expose a better API allowing you to detect that trustability at the callsite).
The bigger design point is who's responsible for knowing where the object lives. Zig has a balance of power more in favour of the caller than the callee, a point I tend to agree with since it's quite easy for a caller to know exactly what they want done and quite hard for a callee to know what all possible callers might want done. That principle is applied pretty liberally through the rest of the language.
To me this kind of stuff always feels a bit cargo cult programming.
however, in languages that don't allow it, you can pick and choose different allocators for different workflows. it may not be important to you, but it is important to many. and shoe horning support for multiple allocators is not easy for language
Zig is a language to use when you do need the control, mostly because of performance reasons. And once you are in, you start thinking how to manage your allocations and arenas are the logical outcome, once you notice that your application has scopes that can be destroyed at once.
If you need an argument why arenas are better than RAII, then you likely don't have the performance needs to even consider using Zig, because outside of performance, arenas are not better.
Languages have opinions, that's why there are several of them and they are popular in some applications and not others. And Zig opinion is to be explicit so RAII is out, and the language does push you into avoiding allocation when unnecessary, as a side effect of not giving you the "easy to use interface" you want.
This makes it particularly suitable for fields like embedded software where you may not have malloc/free at all, but may be less so in a highly dynamic environment, where you may be better served with a GC anyways.
To me, the name "Resource Acquisition Is Initialization" implies that doing one means the other one is done implicitly. Now, in the end of the day, the result is machine language, with proper discipline you can follow the RAII principles in any languages, including assembly, but I have always understood RAII as being it done implicitly.
Also we can argue that Zig supports a weak form of RAII through the use of "defer", there are some nuances here.
You can limit the concurrent requests that can be handled at a time to 2000 and limit memory per request handler to 2MB.
So you allocate 2000 x 2MB at program start.
Then you crate an arena allocator with (2MB limit) when starting to handle a request.
It will just fail if a request uses more memory than 2MB. This means you will basically never have OOM issues or a request using unexpected amount of memory to crash the server.
You also have performance benefits like:
- Allocation operation is basically an integer increment. - Memory for a request is laid out linearly in memory and likely is pre-populated so you won't have page fault into the kernel. - It is very unlikely to have double free or other memory management mistakes.
You can also fuzz the system in several ways to make sure requests can really never exceed that amount of memory usage so you don't have surprises in runtime.
If a request by a user really exceeds that much memory usage, you can just fail that request and log it so it will be debugged and fixed later.
You also don't need data structures like SmallVec (rust version of the thing) that are hybrid stack/heap allocated.
There are many intricacies to this approach but it works much better on scale when you have a large program that handles a large amount of memory.
It is super difficult to debug and fix memory usage issues on global allocator only programs in my experience.
I tried to fix one such issue on a database-like program in rust and it was impossible.
Also this means you don't absolutely need ownership/lifetime safety.
2. Heap allocation is a very common cause of serious performance problems in low-level programs because malloc/free overhead is so high. Some languages have a moving GC to reduce the high overhead of heap allocation, but low-level languages typically can't use that optimisation, so you have to resort to arenas (which offer a similar optimisation to that of moving collectors). This isn't always faster than malloc/free, but it usually is, and if you want malloc/free you can pick that.
Anyway, I wouldn't say that the default in Zig is to use an arena, rather it's to give you the control that's the whole purpose of low-level languages.
[1]: More precisely, the work they need to do doesn't scale linearly with the allocation rate as is the case with non-moving approaches. Of course, arenas are a simple case of a moving collector, so they also display the same effect, but they can't always be used.
> The time to solve performance, and get the 1000x wins, is in the design phase, when you can't profile.
How does this make practical sense? I find it immensely easier to fix performance issues by profiling, benchmarking, finding hotspots. Things that weren't obviously performance bottlenecks during design become obvious when you profile.
Interesting write-up though. I haven't properly tried zig but I like reading these "initial impressions" style articles about it.
That sounds about right, a DS&A course which introduces Hoare's Quicksort, which is old enough to draw a pension, as if that's the state of the art.
Introspective sorting is more than a quarter of a century old and yet probably most people with a CS degree do not even know it exists. What a poor excuse for an engineering discipline this is.
Their big-O studies will lead them to conclude, as a Fortran textbook I've studied did almost forty years ago, that Heapsort is the right choice. Same big-O best case as Quicksort, easier to write but a better worst case. Of course the Fortran textbook does have the heart to mention that Heapsort is actually noticeably slower - but same big-O as if that's what your users will care about.
With a following wind and enough free time (or if they ask an LLM) they might come up with introspective sorting, putting them only a few decades behind the state of the art. Amazing.
I made a PR to add a default zig, which was rejected, I've been using it ever since though.
https://github.com/marler8997/anyzig/pull/71
(about zig's reimplementation of the libc, and some rough edges)
Redoing critical parts such as crypto and compression is crazy in this day and age, hell they even have issues and merge requests open for issues that'd be headliners if found in any major language
https://codeberg.org/ziglang/zig/issues/31131
It's the kind of change that makes me read Zig as not trying to be a serious language. Not that there's anything wrong with that, but the way Zig positions itself is at odds with the completely bizarre takes they do.
> > Won't this get flagged by anti-virus scanners as suspicious?
> Unfortunately, yes. We consider this a problem for the anti-virus scanners to solve.
I am suddenly less interested in relying on Zig.
Zig's decision here now means the problem is _yours_ to convince this third party to trust you.
This is obviously a ridiculous example, but what I'm aiming to say is, if a piece of software is matching something as untrustworthy when it's not, you shouldn't chase it down to force the little guy who's maxing good will software to fix it because the big bully thinks they are evil and deserve to get quarantined
Microsoft breaking the ntdll.dll function interface is extremely unlikely, there's too much popular software in the wild depending on it.
The zig windows NT implementations are very well done and I actually use them as reference when dealing with windows IO.
(joking in case not obvious)
Zig makes handling error easy but makes retaining error information hard. The result is that Zig developers either write more code than other languages to store diagnostic information, or just simply discard diagnostic information. The second makes it much harder to debug because the useful error information is discarded.
About the TigerStyle, it's worth noting that TigerBeetle doesn't even support variable-sized string type. You can store fixed-sized data, such as a string that's at most 32 bytes. What if the order information contains string longer than 32 bytes? Then you need to store that string in antoher database. TigerBeetle is not a general database. Static memory allocation doesn't fit most applicaitons.
The fallable allocation is useful in embedded system and OS development etc., but useless in common application programming, because OOM results in crash rather than failed allocaiton.
I'd bet only a small % of people praising TigerStyle actually disallow all runtime allocation in their own programs.
TB is… extremely forthcoming about this. They’re very explicitly building a specific sort of distributed state machine for high volume transaction data.
If they were building something else they’d end up with a different design. And I suspect a lot of TigerStyle would still apply if that other thing we’re interested in performance or correctness.
FYI, C# is actually C++++ (if you look closely, you can see the four +'s in the octothorpe (C# should be called C-octothorpe; C-sharp would be C♯))
Porting Zig to new operating systems is mostly just porting IO."
Nice!
(In Software Engineering, I think with this change, Zig nailed Separation Of Concerns: (https://en.wikipedia.org/wiki/Separation_of_concerns) aka modularity/modular design, at least with respect to handling the asymmetries of different OS's I/O APIs/syscalls - rather nicely!)
Anyway, very interesting article!
Related:
https://ziglang.org/download/0.16.0/release-notes.html#IO-as...
https://www.ziglang.in/learn/standard-library/io-interface/
In Zig you pass an allocator to every function that allocates, even if it doesn't make a lot of sense.
This rarely makes sense, as it's not really useful to append using a different allocator than you used previously.In Odin, allocators are passed in the init, stored as part of the struct and reused in the append.
The Odin way is useful, too. Depends on what you're building.
It is of course trivial to define your own arraylist type that wraps an allocator on initialisation if you so wish.
One aspect your argument is missing is that if you are juggling many ArrayLists, having each bundle its own copy of the allocator (instead of storing it just once) means wasting 16 bytes per ArrayList. The allocator-less variant is simply more flexible in that regard.
So e.g. that growable array of i64 you've asked for - just the data structure itself, not the array which was the whole point - needs 40 bytes on a modern machine because it has 8 bytes for a pointer to the array, 8 bytes for the size of that array, 8 bytes for how much was used, 8 bytes for a pointer to the allocator procedures and 8 bytes for a (sometimes unused) metadata structure for that allocator.
It's not a huge amount of waste, but what you get for it isn't worth it.
In Zig they're at least not paying for this with runtime space, they're paying for it with the programmer's need to make sure they talk about the same allocator everywhere.
Hypothetically the code is consolidating sites where allocation happens. Zig’s design seems to really push you toward this by making anything else a PITA. So then that allocator reference is being used frequently and already sitting in a register anyway.
The few extra instructions for explicitly passing it might be small compared to the cost of having objects carry their own references. That would grow each instance by ~8 bytes. It would mean that the pointer to the allocator needs to be pulled from a unique memory location for each instance, which is maybe an order of magnitude slower than already having it in a register. Larger data structures would also mean more cache misses, which are another order of magnitude slower.
And this is all chicken feed compared to actually taking a walk of shame out to the DRAM to fetch more memory. So the language ergonomics encouraging you to consolidate your allocation might trump all other considerations in practice.
I haven't looked at the details in Odin, but it's definitely sensible to have the growth decision inline, which means only the (rare) growth itself needs the extra parameter.
I think with Rust 1.100 next month you could write an OdinStyleAllocator type which impl std::alloc::Allocator and has the two pointers inside it, and do the thing Odin does here if you wanted that but I expect you'd snap out of it.
Are you telling me someone was building a tool to switch versions left and right and did not have the thought to name the build file build.zig.zag? Wasted potential
ex: with Zalgebra. Vec3.new doesnt give any autocompletion or error... neiter when you assign it to a variable v = vec3.new(0,0,0); v. you get no autocompletion or variable type you have to read GenericVector and try to understand it... Its like coding in Notepad.