Not all that interesting when you think about it. Doing so would lead to having to admit that the Go team was right that the proposed arena solution isn't right; that there is a better solution out there. Which defies the entire basis of the blog post. The sunk cost fallacy wouldn't want to see all the effort put into the post go to waste upon realizing that the premise is flawed.
The post could have also mentioned that the Go project hasn't given up. There are alternatives being explored to accomplish the same outcome. But, as before, that would invalidate the basis of the post and the sunk cost fallacy cannot stand the idea of having to throw the original premise into the trash.
one question that always plagues me when we talk about mixing manual and automatic memory systems is...how does it work? if we have a mixed graph of automatic and manual objects, it seems like we dont have a choice except to have garbage collection enabled for everything and make a new root (call it the programmer) that keeps track of whether or not the object has been explicitly freed.
since we still have the tracing overhead and the same lifetimes, we haven't really gained that much by having manual memory.
D's best take at this is a compile-time assert that basically forbids us from allocating GC memory in the affected region (please correct me if I'm wrong), but that is pretty limited.
does anyone else have a good narrative for how this would work?
There are some interesting experiments going on in the OCaml world that involve what they call 'modes', essentially a second type system for how a value is used separate from what it is. One goal of modes is to solve this problem. It ends up looking a bit like opting-in to a Rust-style borrow-checker for the relevant functions
> Real-Time Java extends this memory model to support two new kinds of memory: immortal memory and scoped memory. Objects allocated in immortal memory live for the entire execution of the program. The garbage collector scans objects allocated in immortal memory to find (and potentially change) references into the garbage collected heap but does not otherwise manipulate these objects.
> Each scoped memory conceptually contains a preallocated region of memory that threads can enter and exit. Once a thread enters a scoped memory, it can allocate objects out of that memory, with each allocation taking a predictable amount of time. When the thread exits the scoped memory, the implementation deallocates all objects allocated in the scoped memory without garbage collection. The specification supports nested entry and exit of scoped memories, which threads can use to obtain a stack of active scoped memories. The lifetimes of the objects stored in the inner scoped memories are contained in the lifetimes of the objects stored in the outer scoped memories. As for objects allocated in immortal memory, the garbage collector scans objects allocated in scoped memory to find (and potentially change) references into the garbage collected heap but does not otherwise manipulate these objects.
> The Real-Time Java specification uses dynamic access checks to prevent dangling references and ensure the safety of using scoped memories. If the program attempts to create either 1) a reference from an object allocated in the heap to an object allocated in a scoped memory or 2) a reference from an object allocated in an outer scoped memory to an object allocated in an inner scoped
memory, the specification requires the implementation to throw an exception.
>If you choose TypeScript or Python, you’ll hit a performance wall the moment you venture outside of web apps, CRUD servers, and modeling.
This really isn't very accurate. It is for Python, but JavaScript is massively performant. It's so performant that you can write game loops in it provided you work around the garbage collector, which, as noted, is a foible golang shares.
Arenas is one of those patterns that very easy to underestimate. I didn't know about it when I started programming and I run into huge performance issue where I needed to deallocate a huge (sometimes tens of GBs consisting of millions of objects) structure just to make a new one. It was often faster to kill the process and start a new one but that had other downsides. At some point we added a simple hand written arena-like allocator and used it along with malloc. The arena was there for objects on that big structure that will all die at the same point and malloc was for all the other things.
The speed-up was impossible to measure because deallocation that used to take up to 30 seconds (especially after repeat cycles of allocating/deallocating) was now instant.
Even though we had very little experience it was trivial to do in C. Imo it's critical for performance oriented language to make using multiple allocators convenient. GC is a known performance killer but so is malloc in some circumstances.
> By killing Memory Arenas, Go effectively capped its performance ceiling.
I'm still optimistic about potential improvements. (Granted, I doubt there will be anything landing in the near future beyond what the author has already mentioned.)
For example, there is an ongoing discussion on "memory regions" as a successor to the arena concept, without the API "infection" problem:
At the end of the day there has to be a tradeoff between ease of use and performance. Having spent a lot of time optimizing high throughput services in go, it always felt like I was fighting the language. And that's because I was... sure they could add arenas but that just feels like what it is, a patch over the fact you're working alongside a GC.
The vibe I get from this post is of someone who hasn't routinely used arenas in the past and thinks they're kind of a big deal. But a huge part of the point of an arena is how simple it is. You can just build one. Meanwhile, the idea that arena handles were going to be threaded through every high-allocation path in the standard library is fanciful.
I wonder whether it would be possible to retrofit Arena allocation transparently (and safely!) onto a language with a moving GC (which IIUC Go currently is not):
You could ask the programmer to mark some callstack as arena allocated and redirect all allocations to there while active and move everything that is still live once you leave the arena marked callstack (should be cheap if the live set is small, expensive but still safe otherwise).
My guess is that when you measure, an arena is not worth the trouble when you run a generational GC, which essentially uses an arena for the eden space already. And if you have an arena, it's probably very short lived and would otherwise live entirely in eden.
Back in the 1960s, my parents were one of the "first generation" of what we'd call "sport climbers" now. They and their friends climbed all over Scotland, and once they'd done that they climbed in Italy and Austria. They packed everything in, and they packed it all back out, camping, bothying, and bivvying in all conditions.
They and their friends spoke disdainfully of the "short toothbrush brigade". These were the climbers who sawed the handles off their toothbrushes, to save like four grammes in their backpack weight. Massively inconveniencing themselves but they sure were a teaspoon lighter!
This feels like that. Really do you think that playing childish pranks on the garbage collector is going to speed up anything? Pick a faster sorting algorithm or something.
Simple arenas are easy enough to write yourself, even if it does make unidiomatic code as the author points out. Pretty much anything that allocates tons of slices sees a huge performance bump from doing this. I -would- like that ability in an easier fashion.
On the other, hand, new users will abuse arenas and use them everywhere because "I read they are faster", leading to way worse code quality and bugs overall.
I do agree it would become infectious. Once people get addicted to microbenchmarking code and seeing arenas a bit faster in whatever test they are running, they're going to ask that all allocating functions often used (especially everything in http and json) have the ability to use arenas, which may make the code more Zig-like. Not a dig at Zig, but that would either make the language rather unwieldy or double the number of functions in every package as far as I can see.
I would like to see a reference to the place/proposal where Go team has actually rejected the idea of arenas. I have not see this ever in their issues.
I think the deeper issue is that Go's garbage collector is just not performant enough. And at the same time, Go is massively parallel with a shared-everything memory model, so as heaps get bigger, the impact of the imperfect GC becomes more and more noticeable.
Java also had this issue, and they spent decades on tuning collectors. Azul even produced custom hardware for it, at one point in time. I don't think Go needs to go in that direction.
The author is confused about how performance tuning works. Step one, get it right. Step two, see if it's fast enough for the problem at hand.
There is almost never a step three.
But if there is, it's this: Step three: measure.
Now enter a loop of "try something, measure, go to step 2".
Of the things you can try, optimizing GC overhead is but one of many options. Arenas are but one of many options for how to do that.
And the thing about performance optimizations are that they can be intensely local. If you can remove 100% of the allocations on just the happy path inside of one hot loop in your code, then when you loop back to step two, you might find you are done. That does not require an arena allocator with global applicability.
Go gives realistic programmers the right tools to succeed.
And Go's limitations give people like the author plenty of ammunition to fight straw men that don't exist. Tant pis.
Philosophical question, but after reaching critical mass, should languages even aspire to more? I.e. do you risk becoming "master of none"? What's wrong with specialist languages? I.e. best of breed vs best of suite?
I agree with author Go is getting squeezed, but it has its use cases. "COBOL of could native" implies it's not selected for new things, but I reach for it frequently (Go > Java for "enterprise software" backends, Go > others for CLI tools, obviously cloud native / terraform / CI ecosystem, etc.).
However in "best of suite" world, ecosystem interop matters. C <> Go is a pain point. As is WASM <> Go. Both make me reach for Rust.
A better route for something like Go IMO is to move to a compacting collector, this would allow them to move to a bump allocator like Java for super fast allocations and would make deallocation effectively "free" by only moving live objects. They may need to make it generational so they aren't constantly moving long lived objects, but that is a memory vs cpu trade off (could be one more GC flag?). If I recall, the previous objection was because of CGo, which would require pinning (since C wouldn't tolerate moved pointers), but every Go dev I know hates CGo and generally avoids it, plus I see they added "runtime.Pinner" in 1.21 which should solve that I suspect (albeit it would suddenly be required I expect for pointers retained in C). Is anyone aware of what other challenges there are moving to a compacting collector/bump allocator?
> this would allow them to move to a bump allocator like Java for super fast allocations and would make deallocation effectively "free" by only moving live objects
Ah, you've been misled as well. Super fast allocations is a meme. Yes, the very act of allocating is fast, great. Just like tossing trash on the floor. Super fast. Now you have a pile of garbage on the floor that you need to clean up. How fast are you going to end up being after the clean up?
In a design space as tightly constrained as a GC you can't just make something fast. You have to trade off something else to get it. So now that you have sacrificed something to make the act of allocating fast, you've also encouraged programmers using your language to allocate willy-nilly—it's fast after all. Now the pile of garbage is rising at an alarming rate and your self-gimped GC has to deal with it all.
Making allocations fast is a positive feedback loop that degrades GC performance. You want allocations to be slow and to leverage the leeway from that tradeoff to get a faster GC. Moreover, this will provide backpressure to the entire package ecosystem to limit allocations, further improving performance.
> The real reason was the “Infectious API” problem. To get performance benefits, you can’t just create an arena locally; you have to pass it down the call stack so functions can allocate inside it. This forces a rewrite of function signatures.
Sorry, but it doesn't seem that difficult (famous last words). Add a new implicit parameter to all objects just like "this" called "thisArena". When a call to any allocation is made, pass "thisArena" implicitly, unless something else passed explicitly.
That way the arena is viral all the way down and you can create sub-arenas. It also does not require actually passing the arena as parameter.
You don't even need to rewrite any new code, just recompile it.
>Instead of asking the runtime for memory object-by-object, an Arena lets you allocate a large pool of memory upfront. You fill that pool with objects using a simple bump pointer (which is CPU cache-friendly), and when you are done, you free the entire pool at once
>They have been trying to prove they can achieve Arena-like benefits with, for example, improved GC algorithms, but all have failed to land
The new Green Tea GC from Go 1.25 [0]:
Instead of scanning objects we scan whole pages. Instead of tracking objects on our work list, we track whole pages. We still need to mark objects at the end of the day, but we’ll track marked objects locally to each page, rather than across the whole heap.
Sounds like a similar direction: "let's work with many objects at once". They mention better cache-friendliness and all.
I am a bit confused about the API pollution issue with arenas. I think it's a valid point to think about, but at the same time I don't think the average dev will do any extra steps for the faster thing to do.
Going from something like "Go lacks a builtin arena allocation" to "Go risks becoming the COBOL" is a long stretch. First, Go is slower than C/C++/rust without complex memory allocation. Introducing an arena allocator won't fix that. Second, arena allocation often doesn't work for a lot of allocation patterns. Third, plain arena allocator is easy to implement when needed. Surely a builtin one would be better but Go won't fall without it.
45 comments
[ 3.3 ms ] story [ 58.0 ms ] threadThe post could have also mentioned that the Go project hasn't given up. There are alternatives being explored to accomplish the same outcome. But, as before, that would invalidate the basis of the post and the sunk cost fallacy cannot stand the idea of having to throw the original premise into the trash.
since we still have the tracing overhead and the same lifetimes, we haven't really gained that much by having manual memory.
D's best take at this is a compile-time assert that basically forbids us from allocating GC memory in the affected region (please correct me if I'm wrong), but that is pretty limited.
does anyone else have a good narrative for how this would work?
https://oxcaml.org/documentation/modes/intro/
From a quick search, _An Implementation of Scoped Memory for Real-Time Java_ (https://people.csail.mit.edu/rinard/paper/emsoft01.pdf) provides a decent overview:
> Real-Time Java extends this memory model to support two new kinds of memory: immortal memory and scoped memory. Objects allocated in immortal memory live for the entire execution of the program. The garbage collector scans objects allocated in immortal memory to find (and potentially change) references into the garbage collected heap but does not otherwise manipulate these objects.
> Each scoped memory conceptually contains a preallocated region of memory that threads can enter and exit. Once a thread enters a scoped memory, it can allocate objects out of that memory, with each allocation taking a predictable amount of time. When the thread exits the scoped memory, the implementation deallocates all objects allocated in the scoped memory without garbage collection. The specification supports nested entry and exit of scoped memories, which threads can use to obtain a stack of active scoped memories. The lifetimes of the objects stored in the inner scoped memories are contained in the lifetimes of the objects stored in the outer scoped memories. As for objects allocated in immortal memory, the garbage collector scans objects allocated in scoped memory to find (and potentially change) references into the garbage collected heap but does not otherwise manipulate these objects.
> The Real-Time Java specification uses dynamic access checks to prevent dangling references and ensure the safety of using scoped memories. If the program attempts to create either 1) a reference from an object allocated in the heap to an object allocated in a scoped memory or 2) a reference from an object allocated in an outer scoped memory to an object allocated in an inner scoped memory, the specification requires the implementation to throw an exception.
This really isn't very accurate. It is for Python, but JavaScript is massively performant. It's so performant that you can write game loops in it provided you work around the garbage collector, which, as noted, is a foible golang shares.
The solution is the same, to pre-allocate memory.
The speed-up was impossible to measure because deallocation that used to take up to 30 seconds (especially after repeat cycles of allocating/deallocating) was now instant.
Even though we had very little experience it was trivial to do in C. Imo it's critical for performance oriented language to make using multiple allocators convenient. GC is a known performance killer but so is malloc in some circumstances.
I'm still optimistic about potential improvements. (Granted, I doubt there will be anything landing in the near future beyond what the author has already mentioned.)
For example, there is an ongoing discussion on "memory regions" as a successor to the arena concept, without the API "infection" problem:
https://github.com/golang/go/discussions/70257
You could ask the programmer to mark some callstack as arena allocated and redirect all allocations to there while active and move everything that is still live once you leave the arena marked callstack (should be cheap if the live set is small, expensive but still safe otherwise).
They and their friends spoke disdainfully of the "short toothbrush brigade". These were the climbers who sawed the handles off their toothbrushes, to save like four grammes in their backpack weight. Massively inconveniencing themselves but they sure were a teaspoon lighter!
This feels like that. Really do you think that playing childish pranks on the garbage collector is going to speed up anything? Pick a faster sorting algorithm or something.
Simple arenas are easy enough to write yourself, even if it does make unidiomatic code as the author points out. Pretty much anything that allocates tons of slices sees a huge performance bump from doing this. I -would- like that ability in an easier fashion.
On the other, hand, new users will abuse arenas and use them everywhere because "I read they are faster", leading to way worse code quality and bugs overall.
I do agree it would become infectious. Once people get addicted to microbenchmarking code and seeing arenas a bit faster in whatever test they are running, they're going to ask that all allocating functions often used (especially everything in http and json) have the ability to use arenas, which may make the code more Zig-like. Not a dig at Zig, but that would either make the language rather unwieldy or double the number of functions in every package as far as I can see.
I think the deeper issue is that Go's garbage collector is just not performant enough. And at the same time, Go is massively parallel with a shared-everything memory model, so as heaps get bigger, the impact of the imperfect GC becomes more and more noticeable.
Java also had this issue, and they spent decades on tuning collectors. Azul even produced custom hardware for it, at one point in time. I don't think Go needs to go in that direction.
There is almost never a step three.
But if there is, it's this: Step three: measure.
Now enter a loop of "try something, measure, go to step 2".
Of the things you can try, optimizing GC overhead is but one of many options. Arenas are but one of many options for how to do that.
And the thing about performance optimizations are that they can be intensely local. If you can remove 100% of the allocations on just the happy path inside of one hot loop in your code, then when you loop back to step two, you might find you are done. That does not require an arena allocator with global applicability.
Go gives realistic programmers the right tools to succeed.
And Go's limitations give people like the author plenty of ammunition to fight straw men that don't exist. Tant pis.
I agree with author Go is getting squeezed, but it has its use cases. "COBOL of could native" implies it's not selected for new things, but I reach for it frequently (Go > Java for "enterprise software" backends, Go > others for CLI tools, obviously cloud native / terraform / CI ecosystem, etc.).
However in "best of suite" world, ecosystem interop matters. C <> Go is a pain point. As is WASM <> Go. Both make me reach for Rust.
Ah, you've been misled as well. Super fast allocations is a meme. Yes, the very act of allocating is fast, great. Just like tossing trash on the floor. Super fast. Now you have a pile of garbage on the floor that you need to clean up. How fast are you going to end up being after the clean up?
In a design space as tightly constrained as a GC you can't just make something fast. You have to trade off something else to get it. So now that you have sacrificed something to make the act of allocating fast, you've also encouraged programmers using your language to allocate willy-nilly—it's fast after all. Now the pile of garbage is rising at an alarming rate and your self-gimped GC has to deal with it all.
Making allocations fast is a positive feedback loop that degrades GC performance. You want allocations to be slow and to leverage the leeway from that tradeoff to get a faster GC. Moreover, this will provide backpressure to the entire package ecosystem to limit allocations, further improving performance.
Sorry, but it doesn't seem that difficult (famous last words). Add a new implicit parameter to all objects just like "this" called "thisArena". When a call to any allocation is made, pass "thisArena" implicitly, unless something else passed explicitly.
That way the arena is viral all the way down and you can create sub-arenas. It also does not require actually passing the arena as parameter.
You don't even need to rewrite any new code, just recompile it.
>They have been trying to prove they can achieve Arena-like benefits with, for example, improved GC algorithms, but all have failed to land
The new Green Tea GC from Go 1.25 [0]:
Sounds like a similar direction: "let's work with many objects at once". They mention better cache-friendliness and all.[0] https://go.dev/blog/greenteagc