I'm not sure what "memory efficient" means. But, Go sprung as a competitor to Java (portability, language stability, corporate language support/development) and C++ (faster compile times). Can't beat C++ in terms of memory management (performance, guys, not safety) much. But, you can fare well against the JVM, I'm guessing.
In this benchmark actually no, Go doesn't fare well. There is actually higher static overhead per goroutine than JVM VirtualThread.
I presume this is because of a larger initial stack size though/
This probably doesn't matter in the real world as you will actually use the tasks to do some real work which should really dwarf the static overhead is almost all cases.
If you want to run 1 million coroutines that just sleep in your app, yeah nodejs looks very efficient. The problem is that when each coroutine needs to allocate memory, which I would suppose anything real would do, the 2Kb Go pre-allocates will be an advantage - as it will probably be required except for the most trivial workloads (like in this benchmark) - and then because Go actually runs them in parallel, unlike nodejs, you would likely see a huge improvement in both performance and memory usage with Go or Rust.
I write (async) Rust regularly, and I don't understand how the version in the appendix doesn't take 10x1,000,000 seconds to complete. In other words, I'd have expected no concurrency to take place.
Am I wrong?
UPDATE: From the replies below, it looks like I was right about "no concurrency takes place", but I was wrong about how long it takes, because `tokio::time::sleep()` keeps track of when the future was created, (ie when `sleep()` was called) instead of when the future is first `.await`ed (which was my unsaid assumption).
I think the points people made in other replies make sense, but "Tokio::sleep is async" by itself is not enough of an explanation. If it were the case that `Tokio::sleep()` tracked the moment `.await` was called as it's start time, I believe it would indeed take 10x1,000,000 seconds, _even if it's async_.
The implementation of `sleep` [1] decides the wake up time by when `sleep` is called, rather than when its future is polled. So the first task waits one second, then the remaining tasks see that they have already passed the wake-up time and so return instantly.
Yeah, I think you're wrong. It should only take ~10s. tokio::time::sleep records the time it was called before returning the future [1]. So, all 1 million tasks should be stamped with +/- the same time (within a few milliseconds).
> because `tokio::time::sleep()` keeps track of when the future was created, (ie when `sleep()` was called) instead of when the future is first `.await`ed
I’m not a Rust programmer but I strongly suspect this updated explanation is erroneous. It’s probably more like this: start time is recorded when the task execution is started. However, the task immediately yields control back to the async loop. Then the async loop starts another task, and so on. It’s just that the async loop only returns the control to sleeping task no earlier than the moment 1s passes after the task execution was initialy started. I’d be surprised if it had anything to do with when sleep() was called.
On the other hand, I maintain that this is an incidental rather than essential reason for the program finishing quickly. In that benchmark code, we can replace "sleep" with our custom sleep function which does not record start time before execution:
In `wrapped_sleep` function body, where does the `sleep()` come from? It's still tokio::time::sleep, right? If so, the start time is recorded before the first `.await`.
Regardless, the program you provided _does_ actually run the futures concurrently, because of the `join_all()`. My point above was that in the original blog post, the appendix has a version without `join_all()`, which has no concurrency.
> high number of concurrent tasks can consume a significant amount of memory
note absolute numbers here: in the worst case, 1M tasks consumed 2.7 GB of RAM, with ~2700 bytes overhead per task. That'd still fit in the cheapest server with room to spare.
My conclusion would be opposite: as long as per-task data is more than a few KB, the memory overhead of task scheduler is negligible.
Except it’s more than that. Go and Java maintain a stack for every virtual thread. They are clever about it, but it’s very possible that doing anything more than a sleep would have blown up memory on those two systems.
I have a sneaky suspicion if you do anything other than the sleep during these 1 million tasks, you'll blow up memory on all of these systems.
That's kind of the Achille's Heel of the benchmark. Any business needing to spawn 1 million tasks, certainly wants to do something on them. It's the "do something on them" part that usually leads to difficulties for these things. Not really the "spawn a million tasks" part.
The “do something” OP is referring to is simple things like a deeply nested set of function calls and on stack data structures allocated and freed before you sleep. This increases the size of the stack that Go needs to save. By comparison stackless coroutines only save enough information for the continuation, no more no less. That’s going to be strictly smaller than saving the entire stack. The argument you seem to be making is that that could be the same size as the stack (eg heap allocations) but I think that’s being unreasonably optimistic. It should always end up being strictly smaller.
I have no doubt it's going to be strictly smaller, it's just the difference could be too small to care.
Like say you are making a server, and each client has 16KB of state. Then memory usage would be 17KB in Node vs 19 KB in Go. Smaller? Yes. Smaller enough that you want to rewrite the whole app? Probably not.
But it wouldn’t necessarily be 17kib vs 19kib then. It would be reasonable to assume that larger context would be paired with more stack usage so it’s 17kib vs 21 or 25kib and that’s 20-40% more memory being required for Go. That can be quite substantial as 1M concurrent clients represents a case where that memory starts becoming quite a premium.
This depends a lot on how you define "concurrent tasks", but the article provides a definition:
Let's launch N concurrent tasks, where each task waits for 10 seconds and then the program exists after all tasks finish. The number of tasks is controlled by the command line argument.
Leaving aside semantics like "since the tasks aren't specified as doing anything with side effects, the compiler can remove them as dead code", all you really need here is a timer and a continuation for each "task" -- i.e 24 bytes on most platforms. Allowing for allocation overhead and a data structure to manage all the timers efficiently, you might use as much as double that; with some tricks (e.g. function pointer compression) you could get it down to half that.
Eyeballing the graph, it looks like the winner is around 200MB for 1M concurrent tasks, so about 4x worse than a reasonably efficient but not heavily optimized implementation would be.
I have no idea what Go is doing to get 2500 bytes per task.
> I have no idea what Go is doing to get 2500 bytes per task.
TFA creates a goroutine (green thread) for each task (using a waitgroup to synchronise them). IIRC goroutines default to 2k stacks, so that’s about right.
One could argue it’s not fair and it should be timers which would be much lighter. There’s no “efficient wait” for them but that’s essentially the same as the appendix rust program.
If that memory isn't being used and other things need the memory then the OS will very quickly dump it into swap, and as it's never being touched the OS will never need to bring it back in to physical memory. So while it's allocated it doesn't tie up the physical RAM.
You know what I mean. If this was a real world program where those million tasks actually performed work, then this stack space is available for the application to do that work.
It’s not memory that’s consumed by the runtime, it’s memory the runtime expects the program to use - it’s just that this program does no useful work.
I am not u/masklinn - but I don't know what you mean. Doesn't the runtime consume memory by setting it aside for future use? Like what else does "using" ram mean other than claiming it for a time?
I think he means that if the Go code had done something more useful, it would use about the same amount of memory. Compare that to another implementation, which might allocate nearly no memory when the tasks don't do anything significant but would quickly catch up to Go if they did.
If the example was extended to, say, once the sleep is completed then parse and process some JSON data (simulating the sleep being a wait on some remote service), then how would memory use be affected?
In the Go number reported, the majority of the memory is the stack Go allocated for the application code anticipating processing to happen. In the Node example, the processing instead will need heap allocation.
Point being that the two numbers are different - one measures just the overhead of the runtime, the other adds the memory reserved for the app to do work.
The result then looks wasteful for Go because the benchmark.. doesn’t do anything. In a real app though, preallocating stack can often be faster than doing just-in-time heap allocation.
Not always of course! Just noting that the numbers are different things; one is runtime cost, one is runtime cost plus an optimization that assumes memory will be needed for processing after the sleep.
I mean, so I guess you are saying that other languages are better at estimating the memory usage than Go - as go will never need this memory it has allocated? Like Go knows everything that will happen in that goroutine, it should be able to right-size it. I don't think it "looks" wasteful for Go to allocate memory it should know it doesn't need at compile time - I think it is wasteful. Though it's probably not a meaningful waste most of the time.
I agree that it would also be interesting to benchmark some actual stack or heap usage and how the runtimes handle that, but if you are running a massively parallel app you do sometimes end up scheduling jobs to sleep (or perhaps, more likely, to prepare to act but they never do and get cancelled). So I think this is a valid concern, even though it's not the only thing that amtters.
Allocating virtual memory is distinct from actually consuming physical memory (RAM).
If a process allocates many pages of virtual memory, but never actually reads or writes to that memory, then it's unlikely that any physical memory backs those pages. In this sense, allocating memory is really just bookkeeping in the operating system. It's when you try to read or write that memory that the operating system will actually allocate physical memory for you.
When you first try to access the virtual memory you've allocated, there will be a page fault, causing the OS to determine if you're actually allowed to read or write to it. If you've previously allocated it, then all is good, and the OS will allocate some physical memory for you, and make your virtual memory pointers point to that physical memory. If not, well, then that's a segfault. It's not until you first try to use the memory that you actually consume RAM.
I'll yield it would be interesting to have a similar benchmark but instead of sleeping - which indeed by itself is nonsense, to instead each task compute a small fib sequence, or write a small file; something like that.
The argument then, is what if we DO load 2K worth [0] of randomized data into each of those 1m goroutines (and equivalents in the other languages), and do some actual processing. Would we still see the equivalent 10x (whatever math works it out to be) memory "bloat"? And what about performance?
We, as devs, have "4" such resources available to us, memory, network, I/O and compute. And it behooves us to not prematurely optimize on just one.
[0] I can see more arguments/discussions now, "2K is too low, it should be 2MB" etc...!
So the argument is “if you measure something completely different from and unrelated to the article you do not get the same result”?
I guess that’s true.
And to be clear, I do agree with the top comment (which seems to be by you), TFA uses timers in the other runtimes and go does have timers so using goroutines is unwarranted and unfair. And I said as much a few comment up (although I’d forgotten about AfterFunc so I’d have looped and waited on timer.After which would still have been a pessimisation).
And after thinking more about it the article is in also outright lying: technically it’s only measuring tasks in Go, timers are futures / awaitables but they’re not tasks: they’re not independently scheduled units of work, and are pretty much always special cased by runtimes.
Aha, 2k stacks. I figured that stacks would be page size (or more) so 2500 seemed both too small for the thread to have a stack and too large for it to not have a stack.
2k stacks are an interesting design choice though... presumably they're packed, in which case stack overflow is a serious concern. Most threading systems will do something like allocating a single page for the stack but reserving 31 guard pages in case it needs to grow.
A goroutine stack can grow. (EDIT: With stack copying AFAICT... so no virtual pages reserved for a stack to grow... probably some reason for this design?)
Goroutines being go structures, the runtime can cooperate with itself so it doesn't need to do any sort of probing: function prologues can check if there's enough stack space for its frame, and grow the stack if not.
In reality it does use a guard area (technically I think it's more of a redzone? It doesn't cause access errors and functions with known small static frames can use it without checking).
Yeah it’s the drawback, originally it used segmented stacks but that has its own issues.
And it’s probably not the worst issue because deep stacks and stack pointers will mostly be relevant for long running routines which will stabilise their stack use after a while (even if some are likely subject to threshold effects if they’re at the edge, I would not be surprised if some codebases ballasted stacks ahead of time). Also because stack pointers will get promoted to the heap if they escape so the number of stack pointers is not unlimited, and the pointer has to live downwards on the stack.
They are doing different things in this benchmark.
NodeJS has one thread with a very tight loop.
Go actually spawned 1M green threads.
Honestly this benchmark is just noise. Not to say useless in most real world scenarios. Specially because each operation is doing nothing. It would be somewhat useful if they were doing some operation like a DB or HTTP call.
I feel this benchmark compares apples to oranges in some cases.
For example, for node, the author puts a million promises into the runtime event loop and uses `Promise.all` to wait for them all.
This is very different from, say, the Go version where the author creates a million goroutines and puts `waitgroup.Done` as a defer call.
While this might be the idiomatic way of concurrency in the respective languages, it does not account for how goroutines are fundamentally different from promises, and how the runtime does things differently. For JS, there's a single event loop. Counting the JS execution threads, the event loop thread and whatever else the runtime uses for async I/O, the execution model is fundamentally different from Go. Go (if not using `GOMAXPROCS`) spawns an OS thread for every physical thread that your machine has, and then uses a userspace scheduler to distribute goroutines to those threads. It may spawn more OS threads to account for OS threads sleeping on syscalls. Although I don't think the runtime will spawn extra threads in this case.
It also depends on what the "concurrent tasks" (I know, concurrency != parallelism) are. Tasks such as reading a file or doing a network call are better done with something like promises, but CPU-bound tasks are better done with goroutines or Node worker_threads. It would be interesting to see how the memory usage changes when doing async I/O vs CPU-bound tasks concurrently in different languages.
As far as I know there is no way to do Promise like async in go, you HAVE to create a goroutine for each concurrent async task. If this is really the case then I believe the submition is valid.
But I do think that spawning a goroutine just to do a non-blocking task and get its return is kinda wasteful.
You could in theory create your own event loop and then get the exact same behaviour as Promises in Go, but you probably shouldn't. Goroutines are the way to do this in Go, and it wouldn't be useful to benchmark code that would never be written in real life.
I guess what you can do in golang that would be very similar to the rust impl would be this (and could be helpful even in real life, if all you need is a whole lot of timers):
func test2(count int) {
timers := make([]*time.Timer,count)
for idx, _ := range timers {
timers[idx] = time.NewTimer(10 * time.Second)
}
for idx, _ := range timers {
<-timers[idx].C
}
}
This yields to 263552 Maximum resident set size (kbytes) according to /usr/bin/time -v
I'm not sure if I missed it, but I don't see the benchmark specify how the memory was measured, so I assumed the time -v.
The requirement is to run 1 million concurrent tasks.
Of course each language will have a different way of achieving this task each of which will have their unique pros/cons. That's why we have these different languages to begin with.
The accounting here is weird though; Go isn’t using that RAM, it’s expecting the application to. The reason that doesn’t happen is because it’s a micro benchmark that produces no useful work..
The way the results are presented a reader may think the Go memory usage sounds equivalent to the others - boilerplate, ticket-to-play - and then the Go usage sounds super high.
But they are not the same; that memory is in anticipation of a real world program using it
Isn’t that kind of dumb when none of the other languages do this? Apparently allocating memory is really fast? Maybe we should change the test to load 1MB of data in every task?
Most of those languages (excepting Java virtual threads) uses stackless coroutines. Go uses stackful coroutines which allocates some memory upfront for a goroutine to use
Then it is fair to compare the memory usage of a stackful coroutine to a stack less one as they are the idiomatic way to perform async task on each language.
I mean this is subjective, but as long as it’s clear that one number is “this is the memory the runtime itself consumes to solve this problem” and the other number is “this is the runtime memory use and it includes pre-allocated stack space that a real application would then use”, sure
Point being: Someone reading this to choose which runtime will fit their use case needs to be carefully to not assume the numbers measure the same thing. For some real world use cases the pre allocated stack will perform better than the runtimes that instead will do heap allocations.
Of course, as any microbenchmark, the bare results are useless. The numbers can be interesting only if you take the time to understand the implications.
Maybe. But in that case you will need to do something for each of those users, and which languages are good at that might look quite different from this benchmark.
Actually, I think this benchmark did the right thing, that I wish more benchmarks would do. I'm much less interested in what the differences between compilers
are than in what the actual output will be if I ask a professional Go or Node.js dev to solve the same task. (TBF, it would've been better if the task benchmarked was something useful, eg. handling an HTTP request.)
Go heavily encourages a certain kind of programming; JavaScript heavily encourages a different kind; and the article does a great job at showing what the consequences are.
> Go heavily encourages a certain kind of programming;
True, but it really doesn't encourage you to run 1m goroutines with the standard memory setting. Though it's probably fair to run Go wastefully when you're comparing it to Promise.All.
Of course! That's why the article is telling you that some languages (C#, Rust) are better at it than others (Go, Java). Doesn't mean that Go and Java are bad languages! Just that they aren't good to do this thing.
The article is telling us that you can run really inefficient code. Goroutines should be run with worker pools and a buffered channel and it's silly to not do that and then compare it to things like an optimized Rust crate like Tokio.
Well... I'm actually not sure what ideomatic means (English isn't my first language), but it's the standard way of doing it. You'll even find it as step 2 and 3 here: https://go.dev/tour/concurrency/1
> or the best way you can imagine
I would do a lot much more to tune it if you were in a position where you'd know it would run that many "tasks". I think what many non-Go programmers might run into here is that Go doesn't come with any sort of "magic". Instead it comes with a highly opinionated way of doing things. Compare that to C# which comes with a highly optimized CLR and a bunch really excellent libraries which are continuously optimized by Microsoft and you're going to end up with an article like this. The async libraries are maintaining which tasks are running (though Promise.All is obviously also binding a huge amount of memory you don't have to), while the Go example is running 1 million at once.
You'll also notice that there is no benchmark for execution time. With Go you might actually want to pay with memory, though I'd argue that you'd almost never want to run 1 million Goroutines at once.
Though to be fair to this specific author, it looks like they copied the previous benchmarks and then ran it as-is.
The post was edited, previously it just said roughly this part: "step 2 and 3 here: https://go.dev/tour/concurrency/1". Which - as far as I can tell - does not mention worker pools...
You're right. It is using channels and buffers, but you're right.
It's not part of the actual documentation either, at least not exactly: https://go.dev/doc/effective_go#concurrency You will achieve much the same if you follow it, but my answer should have been yes and no as far as being the "standard" Go way.
As far as practicality goes I actually agree with you: if I knew I were trying to do something to the order of 1,000,000 tasks in Go I would probably use a worker pool for this exact reason. I have done this pattern in Go. It is certainly not unidiomatic.
However, it also isn't the obvious way to do 1,000,000 things concurrently in Go. The obvious way to do 1,000,000 things concurrently in Go is to do a for loop and launch a Goroutine for each thing. It is the native unit of task. It is very tightly tied to how I/O works in Go.
If you are trying to do something like a web server, then the calculus changes a lot. In Go, due to the way I/O works, you really can't do much but have a goroutine or two per connection. However, on the other hand, the overhead that goroutines imply starts to look a lot smaller once you put real workloads on each of the millions of tasks.
This benchmark really does tell you something about the performance and overhead of the Go programming language, but it won't necessarily translate to production workloads the way that it seems like it will. In real workloads where the tasks themselves are usually a lot heavier than the constant cost per task, I actually suspect other issues with Go are likely to crop up first (especially in performance critical contexts, latency.) So realistically, it would probably be a bad idea to extrapolate from a benchmark this synthetic to try to determine anything about real world workloads.
Ultimately though, for whatever purpose a synthetic benchmark like this does serve, I think they did the correct thing. I guess I just wonder exactly what the point of it is. Like, the optimized Rust example uses around 0.12 KiB per task. That's extremely cool, but where in the real world are you going to find tasks where the actual state doesn't completely eclipse that metric? Meanwhile, Go is using around 2.64 KiB per task. 22x larger than Rust as it may be, it's still not very much. I think for most real world cases, you would struggle to find too many tasks where the working set per task is actually that small. Of course, if you do, then I'd reckon optimized async Rust will be a true barn-burner at the task, and a lot of those cases where every byte and millisecond counts, Go does often lose. There are many examples.[1]
In many cases Go is far from optimal: Channels, goroutines, the regex engine, various codec implementations in the standard library, etc. are all far from the most optimal implementation you could imagine. However, I feel like they usually do a good job making the performance very sufficient for a wide range of real world tasks. They have made some tradeoffs that a lot of us find very practical and sensible and it makes Go feel like a language you can usually depend on. I think this is especially true in a world where it was already fine when you can run huge websites on Python + Django and other stacks that are relatively much less efficient in memory and CPU usage than Go.
I'll tell you what this benchmark tells me really though: C# is seriously impressive.
I agree with everything you said and I think you contributed a lot to what I said making things much more clear.
> I'll tell you what this benchmark tells me really though: C# is seriously impressive.
The C# team has done some really great work in recent years. I personally hate working with it and it's "magic", but it's certainly in a very good place as far as trusting the CLR to "just work".
Hilariously I also found the Python benchmark to be rather impressive. I was expecting much worse. Not knowing Python well enough, however, makes it hard to really "trust" the benchmark. A talented Python team might be capable of reducing memory usage as much as following every step of the Go concurrency tour would for Go.
Userspace scheduling of Goroutines, virtual stack and non-deterministic pointer type allocation in Go are as much magic if not more, the syntactic sugar of C# is there to get the language out of your way and usually comes at no cost :)
If you do not like the aesthetics of C# and find Elixir or OCaml family tolerable - perhaps try F#? If you use task CEs there you end up with roughly the same performance profile and get to access huge ecosystem making it one of the few FP languages that can be used in production with minimal risk.
> Userspace scheduling of Goroutines, virtual stack and non-deterministic pointer type allocation in Go are as much magic if not more, the syntactic sugar of C# is there to get the language out of your way and usually comes at no cost :)
I don't think C# does it at no cost. I think it's "attachment" to Clean Code makes most C# code bases horrible messes after a while. I know this is a preference thing and that many people will disagree, but I've seen C# code bases that were so complicated to work with that they were actively hindering the development teams ability to meet the business needs. You don't have to write C# that way, but that's what happens in almost every company where I live.
> If you do not like the aesthetics of C# and find Elixir or OCaml family tolerable - perhaps try F#? If you use task CEs there you end up with roughly the same performance profile and get to access huge ecosystem making it one of the few FP languages that can be used in production with minimal risk.
I mean, I don't think I'll ever have to work within the dotnet ecosystem. The way things are going in the green energy and finance sector which is where my career have taken me I'll mostly get to work with Python (with C/Zig) or Go and possibly Java. C# and dotnet is almost exclusively used at stagnant small-medium sized companies and in the consultance business servicing these companies. This is not because of C# or dotnet but more because of the developer landscape. Java is big in "older" organisations because it's what was taught in universities and because it was always good, Go is replacing C#/Java in a lot of newer companies because there are a lot of success stories around it and a lot of the Java developers are retiring. Python is growing really big because a lot of non-swe engineers and accountant types are using it as well as how it's used in ML/AI/Datawarehouse. PHP is big in the web-shop industry and so on. C# manly made it's way into business at places which ran a lot of windows servers. Since organisations rarely change tech stacks in the more "boring" parts of the world, it's not likely to change much.
I don't think dotnet or C# are bad. I write some powershell for azure automation to help IT operations from time to time, but I really don't like working with C# (or Java). I would personally like to work with Rust or more Zig at some point, but it's not like anyone is adopting Rust around here and while Zig can be used for some things in place of C it's not really "production ready" for most things.
Idiomatic is the word the parent was looking for. The base word is idiom.
It was probably the intent of the parent to mean 'making use of the particular features of the language that are not necessarily common to other languages'.
I'm not a programmer, but you appear to give good examples.
I hope I'm not teaching you to suck eggs... {That's an idiom, meaning teaching someone something they're already expert in. Like teaching your Grandma to suck eggs - which weirdly means blowing out the insides of a raw egg. That's done when using the egg to paint; which is a traditional Easter craft.}
I actually did find "idiomatic" when I looked it up, but I honestly still didn't quite grasp it from the cambridge dictionary. Thanks for explaining it in a way I understand.
In programming Idiomatic is used to reference a programming language’s “best practices” and “style guide”. Obviously programming languages can solve problems in many different ways, but they often develop a “correct way” that matches their design or the personality of their influential community members. Following this advice is Ideomatic. Next time your coworker has their style wrong you can say “I don’t think this is Idiomatic” :D
The fundamental problem is there are two kind of sleep function. One that actually sleeps and other that is a actually a timer that just calls certain callback after a desired interval. Promise is just a syntactic sugar on top of second type. Go certainly could call another function after desired interval using `Timer`.
I think better comparison would be wasting CPU for 10 seconds instead of sleep.
But you wouldn't call a million tasks with `Promise.all` in Node, right? That's just not a thing that one does.
Instead, there's usually going to be some queue outside the VM that will leave you with _some_ sort of chunking and otherwise working in smaller, more manageable bits (that might, incidentally, be shaped in ways that the VM can handle in interesting ways).
It's definitely true to say that the "idioamatic" way of handling things is worth going into, but if part of your synthetic benchmark involves doing something quite out of the ordinary, it feels suspicious.
I generally agree that a "real" benchmark here would be nice. It would be interesting if someone could come up with the "minimum viable non-trivial business logic" that people could use for these benchmarks (perhaps coupled with automation tooling to run the benchmarks)
yeah, right? I mean I don't have a dog in this race, just wished we could get into "normal" repros without having to wonder if some magic is kicking in
You could, and the tasks would run concurrently. Node is single threaded so unless you used one of the I/O calls backed by a thread pool, they would all execute sequentially .
But if I have 1 million tasks which spent 10% of their time on CPU-bound codes, intermixed with other IO-bound codes, and I just want throughput and I'm too lazy to use a proper task queue, then why not?
No professional Go programmer would spawn 1M goroutines unless they're sure they have the memory for it (and even then, only if benchmarks indicate it, which is unlikely). Goroutines have a static stack overhead between 2KiB to 8KiB depending on the platform. You'd use a work stealing approach with a reasonable number of goroutines instead. How many are reasonable needs to be tested because it depends on how long each Goroutine spends waiting for I/O or sleeping.
But I can go further than that: No professional programmer should run 1M concurrent tasks on an ordinary CPU no matter which language because it makes no sense if the CPU has several orders of magnitudes less cores. The tasks are not going to run in parallel anyway.
The basis for running 1 million concurrent tasks is to support 1 million active concurrent user connections. They don't need to run in parallel if async is used. As shown, Rust and C# do well. How would you support it in Go?
The servers I use have limits far below 1M active connections, realistically speaking about 60k simultaneously active connections. So I can't really answer that question. However, it's easy to find answers to that question online [1]. Go is not forcing you to spawn Goroutines when you don't really need them. As I said, the correct way in Go is to use worker pools, the size of which depends on measurable performance because it is connected to how much i/o each Goroutine performs and how long it waits on average.
In what world is the cost of 2.5 GB of RAM per 1 million connections an issue? Are you telling me there's a service in this world handling, for example, 100 million active connections, and they can't afford 250 GB of RAM? It's not the 90's anymore.
And we're talking about a naive microbenchmark. If you were actually building a service like that in Go (millions of active connections) and you were very concerned about memory usage, you wouldn't be naive enough to use a goroutine for every connection. Instead, you would use something like gnet or another solution based directly on epoll events, combined with a worker pool.
Go is a friendly language and it is well liked, but it is inappropriate when the absolute best performance saves more money than the additional developer salaries needed to write it in a higher performance language. An example is for deep learning or other big numerical work, where you'd be wasting expensive hardware resources if using Go. Perhaps the one million concurrent users will however be fine with Go.
Also, for Java, Virtual Threads are a very new feature (Java 21 IIRC or somewhere around there). OS threads have been around for decades. As a heavy JVM user it would have been nice to actually see those both broken out to compare as well!
In the titular post there's a link to a previous comparison between approaches, and plain OS threads used from Rust fare quite well, even if the author doesn't up the OS limits to keep that in the running for the higher thread cases: https://pkolaczk.github.io/memory-consumption-of-async/
What’s missing here is that all these async/await, lightweight threads, etc features exist because presumably because OS processes and threads consume too many resources to feasibly serve a similar role.
However nobody seems to have any hard numbers about the topic. How bad is a Linux context switch? How much memory does a process use? Suddenly everyone is a micro optimizer seeking to gain a small constant multiple.
Since this is not ready at hand I suspect the rewards are much less clear. It’s more likely that the languages benefit from greater cross platform control and want to avoid cross platform inconsistencies.
* gunicorn doesn't have worker scaling, they're all always running, so async workers were to not waste those resources idling while still allowing lots of simultaneous clients
* The limit was 32768 not that long ago, which was at least possible to hit with a ton of clients
Why would I be worried about processes using resources while blocked (not using CPU)? Virtual memory is a thing. And if it’s a web server that’s just not being used.
RAM, not CPU, which goes right back to this post's benchmark. Python code takes up memory too, not just data. The workers consume this memory on startup, not just while handling a request.
Note 1: The gist is in Ukrainian, and the blog post by Steve does a much better job, but hopefully you will find this useful. Feel free to replicate the results and post them.
Note 2: The absolute numbers do not necessarily imply good/bad. Both Go and BEAM focus on userspace scheduling and its fairness. Stackful coroutines have their own advantages. I think where the blog post's data is most relevant is understanding the advantages of stackless coroutines when it comes to "highly granular" concurrency - dispatching concurrent requests, fanning out to process many small items at once, etc. In any case, I did not expect sibling comments to go onto praising Node.js, is it really that surprising for event loop based concurrency? :)
Also, if you are an Elixir aficionado and were impressed by C#'s numbers - know that they translate ~1:1 to F# now that it has task CE, just sayin'.
Here's how the program looks in F#:
open System
open System.Threading.Tasks
let argv = Environment.GetCommandLineArgs()
[1..int argv[1]]
|> Seq.map (fun _ -> Task.Delay(TimeSpan.FromSeconds 10.0))
|> Task.WaitAll
Note that the Task library in Elixir uses supervised processes so it adds a lot more overhead. It would be interesting to see the benchmark with just normal Erlang processes.
It doesn't take that much space, and not all languages have option to easily map an initial range onto an iterator that produces tasks. Most are dominated by the size of state machines/virtual threads.
Please note that the link above leads to old code from .NET Framework.
The referencesource repository is only relevant if you are using the legacy .NET Framework. Modern .NET has a special case for passing a List<Task> and avoids the allocation:
It would be nice if the author also compared different runtimes (e.g. NodeJS vs Deno, or cpython vs pypy) and core language engines (e.g. v8 vs spider monkey vs JavaScript core)
This benchmark is nonsense. Apart from the fact that Go has an average Goroutine overhead of a 4kB stack (meaning an average usage of 3.9GB for 1M tasks), the code written is also in a closure, and scheduling a 2nd Goroutine in the wg.Done(), so unlike some of the others it had at least 2M function calls on the event loop stack in addition to at least 1M closure references. So yeah, it’s a great example of bad code in any language.
This test is no real.
These languages works differently. Garbage collected and manually allocating and deallocating. If you do not configure a garbage collected language correctly it will spin out of control in memory consumption because it will just not garbage collect. If you would have configured the garbage collection to be low for java and go then go would look like rust.
Maybe I’m missing something here but surely Node isn’t doing anything concurrently? Promises don’t execute concurrently, they just tidy up async execution. The code as given will just sequentially resolve a million promises. No wonder it looks so good. You’d need to be using workers to actually do anything concurrently.
Fair point. Kind of makes the comparisons between languages a little unfair, though. Go and Rust would be executing these operations in parallel, Node would not. Would make a significant difference to real world performance!
The measurement is memory, not performance. Paused/queued tasks in the sequential node version still count, and in theory could be worse since the Go and Rust ones would be consuming them in parallel and not building up the queue as much.
But, then they can measure memory by simply using a threat pool of size 1 and then submitting tasks to it right ? That would be the equivalent comparison for other languages.
> Now Go loses by over 13 times to the winner. It also loses by over 2 times to Java, which contradicts the general perception of the JVM being a memory hog and Go being lightweight.
Well, if it isn't the classic unwavering confidence that an artificial "hello world"-like benchmark is in any way representative of real world programs.
Yes, but also, languages like Java and C# have caught up a great deal over the past 10 years and run incredibly smoothly. Most peoples' perception of them being slow is really just from legacy tech that they encountered a long time ago, or (oof) being exposed to some terrible piece of .NET Framework code that's still running on an underprovisioned IIS server.
Regarding Java I'm pretty sure that benchmark is broken at least a little bit and testing something else as not specifying initial size for ArrayList means list of size 10 which gets resized all the time when `add()` is called, leading to big amount of unused objects needing garbage collection.
Yeah that is a junior mistake... They should've pre-sized the ArrayList, or better, used an array because that's more memory efficient (and I would say would be what any decent dev would do when the size of tasks is known beforehand).
> Some folks pointed out that in Rust (tokio) it can use a loop iterating over the Vec instead of join_all to avoid the resize to the list
Right, but some folks also pointed out you should've used an array in Java in the previous blog post, 2 years ago, and you didn't do that.
It can be a big difference if boxing is involved. Or if the list is very big, because all access to items in the list require casting at the bytecode level (due to type erasure).
It would indeed be better to create appropriately sized storage.
However, I don't think that underlying array is resized every time `add` is called. I'd expect that resize will happen less than 30 times for 1M adds (capacity grows geometrically with a=10 and r=1.5)
From the docs: "The add operation runs in amortized constant time, that is, adding n elements requires O(n) time".
Given the linear time time complexity, it seems obvious that adding a thread pointer to the list won't contribute substantially to the thread creation time.
Can we do something more real world at least? what's the cost (hetzner monthly) of maintaining 1M concurrent websocket connection where each make a query to a postgres db randomly every 1-4 seconds.
The cost wouldn't be just Memory because the network card and CPU also enter the game.
Reminder that Rust does not automatically schedule anything. Unless you _explicitly_ call `tokio::spawn` or `async_std::spawn` you are still living entirely in state-machine land.
Rust's `join_all` uses `FuturesUnordered` behind the scenes, which is pretty intelligent in terms of keeping track of which tasks are ready to make progress, but it does not use tokio/async_std for scheduling. AFAICT the only thing being measured about tokio/async_std is the heap size of their `sleep` implementations.
I'd be very interesting in seeing how Tokio's actual scheduler performs. The two ways to do that are:
- spawn each future in the global scheduler, and then await the JoinHandles using the for loop from the appendix
As other commenters have noted, calling `sleep` only constructs a state machine. So the Appendix isn't actually concurrent. Again, you need to either put those state machines into the tokio/async_std schedulers with `spawn`, or combine the state machines with `FuturesUnordered`.
I did this measurement, and using time -v, the maximum resident size in KB comes out to 440,424 kb for 1m tasks, 46,820 kb for 100k, and 7,156 kb for 10k.
It would be more interesting to see a benchmark where a task will not be empty but would have an open network connection e.g. would make an HTTP request to a test server with 10 seconds response time. Network is a frequent reason real world applications spawn 1M tasks.
204 comments
[ 5.5 ms ] story [ 265 ms ] threadI must be missing something - isn’t Go supposed to be memory efficient? Perhaps promises and goroutines aren’t comparable?
This probably doesn't matter in the real world as you will actually use the tasks to do some real work which should really dwarf the static overhead is almost all cases.
Am I wrong?
UPDATE: From the replies below, it looks like I was right about "no concurrency takes place", but I was wrong about how long it takes, because `tokio::time::sleep()` keeps track of when the future was created, (ie when `sleep()` was called) instead of when the future is first `.await`ed (which was my unsaid assumption).
[1]: https://docs.rs/tokio/latest/tokio/time/fn.sleep.html
[1]: https://docs.rs/tokio/1.41.1/src/tokio/time/sleep.rs.html#12...
I’m not a Rust programmer but I strongly suspect this updated explanation is erroneous. It’s probably more like this: start time is recorded when the task execution is started. However, the task immediately yields control back to the async loop. Then the async loop starts another task, and so on. It’s just that the async loop only returns the control to sleeping task no earlier than the moment 1s passes after the task execution was initialy started. I’d be surprised if it had anything to do with when sleep() was called.
On the other hand, I maintain that this is an incidental rather than essential reason for the program finishing quickly. In that benchmark code, we can replace "sleep" with our custom sleep function which does not record start time before execution:
The following program will still finish in ~10 seconds.Regardless, the program you provided _does_ actually run the futures concurrently, because of the `join_all()`. My point above was that in the original blog post, the appendix has a version without `join_all()`, which has no concurrency.
note absolute numbers here: in the worst case, 1M tasks consumed 2.7 GB of RAM, with ~2700 bytes overhead per task. That'd still fit in the cheapest server with room to spare.
My conclusion would be opposite: as long as per-task data is more than a few KB, the memory overhead of task scheduler is negligible.
That's kind of the Achille's Heel of the benchmark. Any business needing to spawn 1 million tasks, certainly wants to do something on them. It's the "do something on them" part that usually leads to difficulties for these things. Not really the "spawn a million tasks" part.
Like say you are making a server, and each client has 16KB of state. Then memory usage would be 17KB in Node vs 19 KB in Go. Smaller? Yes. Smaller enough that you want to rewrite the whole app? Probably not.
Let's launch N concurrent tasks, where each task waits for 10 seconds and then the program exists after all tasks finish. The number of tasks is controlled by the command line argument.
Leaving aside semantics like "since the tasks aren't specified as doing anything with side effects, the compiler can remove them as dead code", all you really need here is a timer and a continuation for each "task" -- i.e 24 bytes on most platforms. Allowing for allocation overhead and a data structure to manage all the timers efficiently, you might use as much as double that; with some tricks (e.g. function pointer compression) you could get it down to half that.
Eyeballing the graph, it looks like the winner is around 200MB for 1M concurrent tasks, so about 4x worse than a reasonably efficient but not heavily optimized implementation would be.
I have no idea what Go is doing to get 2500 bytes per task.
https://github.com/golang/go/blob/master/src/runtime/stack.g...
TFA creates a goroutine (green thread) for each task (using a waitgroup to synchronise them). IIRC goroutines default to 2k stacks, so that’s about right.
One could argue it’s not fair and it should be timers which would be much lighter. There’s no “efficient wait” for them but that’s essentially the same as the appendix rust program.
If that memory isn't being used and other things need the memory then the OS will very quickly dump it into swap, and as it's never being touched the OS will never need to bring it back in to physical memory. So while it's allocated it doesn't tie up the physical RAM.
It’s not memory that’s consumed by the runtime, it’s memory the runtime expects the program to use - it’s just that this program does no useful work.
In the Go number reported, the majority of the memory is the stack Go allocated for the application code anticipating processing to happen. In the Node example, the processing instead will need heap allocation.
Point being that the two numbers are different - one measures just the overhead of the runtime, the other adds the memory reserved for the app to do work.
The result then looks wasteful for Go because the benchmark.. doesn’t do anything. In a real app though, preallocating stack can often be faster than doing just-in-time heap allocation.
Not always of course! Just noting that the numbers are different things; one is runtime cost, one is runtime cost plus an optimization that assumes memory will be needed for processing after the sleep.
I agree that it would also be interesting to benchmark some actual stack or heap usage and how the runtimes handle that, but if you are running a massively parallel app you do sometimes end up scheduling jobs to sleep (or perhaps, more likely, to prepare to act but they never do and get cancelled). So I think this is a valid concern, even though it's not the only thing that amtters.
If a process allocates many pages of virtual memory, but never actually reads or writes to that memory, then it's unlikely that any physical memory backs those pages. In this sense, allocating memory is really just bookkeeping in the operating system. It's when you try to read or write that memory that the operating system will actually allocate physical memory for you.
When you first try to access the virtual memory you've allocated, there will be a page fault, causing the OS to determine if you're actually allowed to read or write to it. If you've previously allocated it, then all is good, and the OS will allocate some physical memory for you, and make your virtual memory pointers point to that physical memory. If not, well, then that's a segfault. It's not until you first try to use the memory that you actually consume RAM.
We, as devs, have "4" such resources available to us, memory, network, I/O and compute. And it behooves us to not prematurely optimize on just one.
[0] I can see more arguments/discussions now, "2K is too low, it should be 2MB" etc...!
I guess that’s true.
And to be clear, I do agree with the top comment (which seems to be by you), TFA uses timers in the other runtimes and go does have timers so using goroutines is unwarranted and unfair. And I said as much a few comment up (although I’d forgotten about AfterFunc so I’d have looped and waited on timer.After which would still have been a pessimisation).
And after thinking more about it the article is in also outright lying: technically it’s only measuring tasks in Go, timers are futures / awaitables but they’re not tasks: they’re not independently scheduled units of work, and are pretty much always special cased by runtimes.
2k stacks are an interesting design choice though... presumably they're packed, in which case stack overflow is a serious concern. Most threading systems will do something like allocating a single page for the stack but reserving 31 guard pages in case it needs to grow.
In reality it does use a guard area (technically I think it's more of a redzone? It doesn't cause access errors and functions with known small static frames can use it without checking).
I guess if you're already doing garbage collection moving the stack doesn't make things all that much worse though... still, yuck.
And it’s probably not the worst issue because deep stacks and stack pointers will mostly be relevant for long running routines which will stabilise their stack use after a while (even if some are likely subject to threshold effects if they’re at the edge, I would not be surprised if some codebases ballasted stacks ahead of time). Also because stack pointers will get promoted to the heap if they escape so the number of stack pointers is not unlimited, and the pointer has to live downwards on the stack.
NodeJS has one thread with a very tight loop.
Go actually spawned 1M green threads.
Honestly this benchmark is just noise. Not to say useless in most real world scenarios. Specially because each operation is doing nothing. It would be somewhat useful if they were doing some operation like a DB or HTTP call.
For example, for node, the author puts a million promises into the runtime event loop and uses `Promise.all` to wait for them all.
This is very different from, say, the Go version where the author creates a million goroutines and puts `waitgroup.Done` as a defer call.
While this might be the idiomatic way of concurrency in the respective languages, it does not account for how goroutines are fundamentally different from promises, and how the runtime does things differently. For JS, there's a single event loop. Counting the JS execution threads, the event loop thread and whatever else the runtime uses for async I/O, the execution model is fundamentally different from Go. Go (if not using `GOMAXPROCS`) spawns an OS thread for every physical thread that your machine has, and then uses a userspace scheduler to distribute goroutines to those threads. It may spawn more OS threads to account for OS threads sleeping on syscalls. Although I don't think the runtime will spawn extra threads in this case.
It also depends on what the "concurrent tasks" (I know, concurrency != parallelism) are. Tasks such as reading a file or doing a network call are better done with something like promises, but CPU-bound tasks are better done with goroutines or Node worker_threads. It would be interesting to see how the memory usage changes when doing async I/O vs CPU-bound tasks concurrently in different languages.
But I do think that spawning a goroutine just to do a non-blocking task and get its return is kinda wasteful.
I'm not sure if I missed it, but I don't see the benchmark specify how the memory was measured, so I assumed the time -v.
Of course each language will have a different way of achieving this task each of which will have their unique pros/cons. That's why we have these different languages to begin with.
The way the results are presented a reader may think the Go memory usage sounds equivalent to the others - boilerplate, ticket-to-play - and then the Go usage sounds super high.
But they are not the same; that memory is in anticipation of a real world program using it
Point being: Someone reading this to choose which runtime will fit their use case needs to be carefully to not assume the numbers measure the same thing. For some real world use cases the pre allocated stack will perform better than the runtimes that instead will do heap allocations.
Apart from i/o, allocating memory is usually the slowest thing you can do on a computer, in my experience.
That's not a real requirement though. No business actually needs to run 1 million concurrent tasks with no concern for what's in them.
Go heavily encourages a certain kind of programming; JavaScript heavily encourages a different kind; and the article does a great job at showing what the consequences are.
True, but it really doesn't encourage you to run 1m goroutines with the standard memory setting. Though it's probably fair to run Go wastefully when you're comparing it to Promise.All.
Well... I'm actually not sure what ideomatic means (English isn't my first language), but it's the standard way of doing it. You'll even find it as step 2 and 3 here: https://go.dev/tour/concurrency/1
> or the best way you can imagine
I would do a lot much more to tune it if you were in a position where you'd know it would run that many "tasks". I think what many non-Go programmers might run into here is that Go doesn't come with any sort of "magic". Instead it comes with a highly opinionated way of doing things. Compare that to C# which comes with a highly optimized CLR and a bunch really excellent libraries which are continuously optimized by Microsoft and you're going to end up with an article like this. The async libraries are maintaining which tasks are running (though Promise.All is obviously also binding a huge amount of memory you don't have to), while the Go example is running 1 million at once.
You'll also notice that there is no benchmark for execution time. With Go you might actually want to pay with memory, though I'd argue that you'd almost never want to run 1 million Goroutines at once.
Though to be fair to this specific author, it looks like they copied the previous benchmarks and then ran it as-is.
It's not part of the actual documentation either, at least not exactly: https://go.dev/doc/effective_go#concurrency You will achieve much the same if you follow it, but my answer should have been yes and no as far as being the "standard" Go way.
As far as practicality goes I actually agree with you: if I knew I were trying to do something to the order of 1,000,000 tasks in Go I would probably use a worker pool for this exact reason. I have done this pattern in Go. It is certainly not unidiomatic.
However, it also isn't the obvious way to do 1,000,000 things concurrently in Go. The obvious way to do 1,000,000 things concurrently in Go is to do a for loop and launch a Goroutine for each thing. It is the native unit of task. It is very tightly tied to how I/O works in Go.
If you are trying to do something like a web server, then the calculus changes a lot. In Go, due to the way I/O works, you really can't do much but have a goroutine or two per connection. However, on the other hand, the overhead that goroutines imply starts to look a lot smaller once you put real workloads on each of the millions of tasks.
This benchmark really does tell you something about the performance and overhead of the Go programming language, but it won't necessarily translate to production workloads the way that it seems like it will. In real workloads where the tasks themselves are usually a lot heavier than the constant cost per task, I actually suspect other issues with Go are likely to crop up first (especially in performance critical contexts, latency.) So realistically, it would probably be a bad idea to extrapolate from a benchmark this synthetic to try to determine anything about real world workloads.
Ultimately though, for whatever purpose a synthetic benchmark like this does serve, I think they did the correct thing. I guess I just wonder exactly what the point of it is. Like, the optimized Rust example uses around 0.12 KiB per task. That's extremely cool, but where in the real world are you going to find tasks where the actual state doesn't completely eclipse that metric? Meanwhile, Go is using around 2.64 KiB per task. 22x larger than Rust as it may be, it's still not very much. I think for most real world cases, you would struggle to find too many tasks where the working set per task is actually that small. Of course, if you do, then I'd reckon optimized async Rust will be a true barn-burner at the task, and a lot of those cases where every byte and millisecond counts, Go does often lose. There are many examples.[1]
In many cases Go is far from optimal: Channels, goroutines, the regex engine, various codec implementations in the standard library, etc. are all far from the most optimal implementation you could imagine. However, I feel like they usually do a good job making the performance very sufficient for a wide range of real world tasks. They have made some tradeoffs that a lot of us find very practical and sensible and it makes Go feel like a language you can usually depend on. I think this is especially true in a world where it was already fine when you can run huge websites on Python + Django and other stacks that are relatively much less efficient in memory and CPU usage than Go.
I'll tell you what this benchmark tells me really though: C# is seriously impressive.
[1]: https://discord.com/blog/why-discord-is-switching-from-go-to...
> I'll tell you what this benchmark tells me really though: C# is seriously impressive.
The C# team has done some really great work in recent years. I personally hate working with it and it's "magic", but it's certainly in a very good place as far as trusting the CLR to "just work".
Hilariously I also found the Python benchmark to be rather impressive. I was expecting much worse. Not knowing Python well enough, however, makes it hard to really "trust" the benchmark. A talented Python team might be capable of reducing memory usage as much as following every step of the Go concurrency tour would for Go.
If you do not like the aesthetics of C# and find Elixir or OCaml family tolerable - perhaps try F#? If you use task CEs there you end up with roughly the same performance profile and get to access huge ecosystem making it one of the few FP languages that can be used in production with minimal risk.
I don't think C# does it at no cost. I think it's "attachment" to Clean Code makes most C# code bases horrible messes after a while. I know this is a preference thing and that many people will disagree, but I've seen C# code bases that were so complicated to work with that they were actively hindering the development teams ability to meet the business needs. You don't have to write C# that way, but that's what happens in almost every company where I live.
> If you do not like the aesthetics of C# and find Elixir or OCaml family tolerable - perhaps try F#? If you use task CEs there you end up with roughly the same performance profile and get to access huge ecosystem making it one of the few FP languages that can be used in production with minimal risk.
I mean, I don't think I'll ever have to work within the dotnet ecosystem. The way things are going in the green energy and finance sector which is where my career have taken me I'll mostly get to work with Python (with C/Zig) or Go and possibly Java. C# and dotnet is almost exclusively used at stagnant small-medium sized companies and in the consultance business servicing these companies. This is not because of C# or dotnet but more because of the developer landscape. Java is big in "older" organisations because it's what was taught in universities and because it was always good, Go is replacing C#/Java in a lot of newer companies because there are a lot of success stories around it and a lot of the Java developers are retiring. Python is growing really big because a lot of non-swe engineers and accountant types are using it as well as how it's used in ML/AI/Datawarehouse. PHP is big in the web-shop industry and so on. C# manly made it's way into business at places which ran a lot of windows servers. Since organisations rarely change tech stacks in the more "boring" parts of the world, it's not likely to change much.
I don't think dotnet or C# are bad. I write some powershell for azure automation to help IT operations from time to time, but I really don't like working with C# (or Java). I would personally like to work with Rust or more Zig at some point, but it's not like anyone is adopting Rust around here and while Zig can be used for some things in place of C it's not really "production ready" for most things.
It was probably the intent of the parent to mean 'making use of the particular features of the language that are not necessarily common to other languages'.
I'm not a programmer, but you appear to give good examples.
I hope I'm not teaching you to suck eggs... {That's an idiom, meaning teaching someone something they're already expert in. Like teaching your Grandma to suck eggs - which weirdly means blowing out the insides of a raw egg. That's done when using the egg to paint; which is a traditional Easter craft.}
I think better comparison would be wasting CPU for 10 seconds instead of sleep.
Instead, there's usually going to be some queue outside the VM that will leave you with _some_ sort of chunking and otherwise working in smaller, more manageable bits (that might, incidentally, be shaped in ways that the VM can handle in interesting ways).
It's definitely true to say that the "idioamatic" way of handling things is worth going into, but if part of your synthetic benchmark involves doing something quite out of the ordinary, it feels suspicious.
I generally agree that a "real" benchmark here would be nice. It would be interesting if someone could come up with the "minimum viable non-trivial business logic" that people could use for these benchmarks (perhaps coupled with automation tooling to run the benchmarks)
But neither would you wait on a waitgroup of size 1 million in Go... right?
But if I have 1 million tasks which spent 10% of their time on CPU-bound codes, intermixed with other IO-bound codes, and I just want throughput and I'm too lazy to use a proper task queue, then why not?
But I can go further than that: No professional programmer should run 1M concurrent tasks on an ordinary CPU no matter which language because it makes no sense if the CPU has several orders of magnitudes less cores. The tasks are not going to run in parallel anyway.
[1] https://www.freecodecamp.org/news/million-websockets-and-go-...
And we're talking about a naive microbenchmark. If you were actually building a service like that in Go (millions of active connections) and you were very concerned about memory usage, you wouldn't be naive enough to use a goroutine for every connection. Instead, you would use something like gnet or another solution based directly on epoll events, combined with a worker pool.
Or, much more likely, pay a few tens of dollars per month for a second server and start scaling out.
In the titular post there's a link to a previous comparison between approaches, and plain OS threads used from Rust fare quite well, even if the author doesn't up the OS limits to keep that in the running for the higher thread cases: https://pkolaczk.github.io/memory-consumption-of-async/
What’s missing here is that all these async/await, lightweight threads, etc features exist because presumably because OS processes and threads consume too many resources to feasibly serve a similar role.
However nobody seems to have any hard numbers about the topic. How bad is a Linux context switch? How much memory does a process use? Suddenly everyone is a micro optimizer seeking to gain a small constant multiple.
Since this is not ready at hand I suspect the rewards are much less clear. It’s more likely that the languages benefit from greater cross platform control and want to avoid cross platform inconsistencies.
* gunicorn doesn't have worker scaling, they're all always running, so async workers were to not waste those resources idling while still allowing lots of simultaneous clients
* The limit was 32768 not that long ago, which was at least possible to hit with a ton of clients
Note 1: The gist is in Ukrainian, and the blog post by Steve does a much better job, but hopefully you will find this useful. Feel free to replicate the results and post them.
Note 2: The absolute numbers do not necessarily imply good/bad. Both Go and BEAM focus on userspace scheduling and its fairness. Stackful coroutines have their own advantages. I think where the blog post's data is most relevant is understanding the advantages of stackless coroutines when it comes to "highly granular" concurrency - dispatching concurrent requests, fanning out to process many small items at once, etc. In any case, I did not expect sibling comments to go onto praising Node.js, is it really that surprising for event loop based concurrency? :)
Also, if you are an Elixir aficionado and were impressed by C#'s numbers - know that they translate ~1:1 to F# now that it has task CE, just sayin'.
Here's how the program looks in F#:
Souce: https://github.com/microsoft/referencesource/blob/master/msc...
Please note that the link above leads to old code from .NET Framework.
The up-to-date .NET implementation lives here: https://github.com/dotnet/runtime/blob/main/src/libraries/Sy...
https://github.com/dotnet/runtime/blob/1f01bee2a41e0df97089f...
But, then they can measure memory by simply using a threat pool of size 1 and then submitting tasks to it right ? That would be the equivalent comparison for other languages.
They should launch a million NodeJS processes.
Well, if it isn't the classic unwavering confidence that an artificial "hello world"-like benchmark is in any way representative of real world programs.
> Some folks pointed out that in Rust (tokio) it can use a loop iterating over the Vec instead of join_all to avoid the resize to the list
Right, but some folks also pointed out you should've used an array in Java in the previous blog post, 2 years ago, and you didn't do that.
And folks also pointed out Elixir shouldn't have used Task in the previous benchmark (folk here being the creator of Elixir himself): https://github.com/pkolaczk/async-runtimes-benchmarks/pull/7
However, I don't think that underlying array is resized every time `add` is called. I'd expect that resize will happen less than 30 times for 1M adds (capacity grows geometrically with a=10 and r=1.5)
Given the linear time time complexity, it seems obvious that adding a thread pointer to the list won't contribute substantially to the thread creation time.
The way these kinds of operations can be implemented for amortized linear time (also in Python, C realloc, etc) is explained in https://en.wikipedia.org/wiki/Dynamic_array#Geometric_expans...
The cost wouldn't be just Memory because the network card and CPU also enter the game.
Rust's `join_all` uses `FuturesUnordered` behind the scenes, which is pretty intelligent in terms of keeping track of which tasks are ready to make progress, but it does not use tokio/async_std for scheduling. AFAICT the only thing being measured about tokio/async_std is the heap size of their `sleep` implementations.
I'd be very interesting in seeing how Tokio's actual scheduler performs. The two ways to do that are:
- using https://docs.rs/tokio/latest/tokio/task/join_set/struct.Join... to spawn all the futures and then await them
- spawn each future in the global scheduler, and then await the JoinHandles using the for loop from the appendix
As other commenters have noted, calling `sleep` only constructs a state machine. So the Appendix isn't actually concurrent. Again, you need to either put those state machines into the tokio/async_std schedulers with `spawn`, or combine the state machines with `FuturesUnordered`.