> Used pervasively, Arc gives you the world’s worst garbage collector. Like a GC, the lifetime of objects and the resources they represent (memory, files, sockets) is unknowable. But you take this loss without the wins you’d get from an actual GC!
The lifetime of an Arc isn’t unknowable, it’s determined by where and how you hold it.
I think maybe the disconnect in this article is that the author is coming at Rust and trying to force their previous mental models on to it (such as garbage collection) rather than learning how to work with the language. It’s a common trap for anyone trying a new programming language, but Rust seems to trip people up more than most.
> The lifetime of an Arc isn’t unknowable, it’s determined by where and how you hold it.
In the same sense that the lifetime of an object in a GC'd system has a lower bound of, "as long as it's referenced", sure. But that's nearly the opposite of what the borrow checker tries to do by statically bounding objects, at compile time.
> maybe the disconnect in this article is that the author is coming at Rust and trying to force their previous mental models on to it
The opposite actually! I spent about a decade doing systems programming in C, C++, and Rust before writing a bunch of Haskell at my current job. The degree to which a big language runtime and GC weren't a boogeyman for some problem spaces was really eye-opening.
Honestly, the biggest stumbling block for rust and async is the notion of memory pinning.
Rust will do a lot of invisible memory relocations under the covers. Which can work great in single threaded contexts. However, once you start talking about threading those invisible memory moves are a hazard. The moment shared memory comes into play everything just gets a whole lot harder with the rust async story.
Contrast that with a language like java or go. It's true that the compiler won't catch you when 2 threads access the same shared memory, but at the same time the mental burden around "Where is this in memory, how do I make sure it deallocates correctly, etc" just evaporates. A whole host of complex types are erased and the language simply cleans up stuff when nothing references it.
To me, it seems like GCs simply make a language better for concurrency. They generally solve a complex problem.
> Rust will do a lot of invisible memory relocations under the covers.
I don't think it's quite accurate to point to "invisible memory relocations" as the problem that pinning solves. In most cases, memory relocations in Rust are very explicit, by moving an owned value when it has no live references (if it has any references, the borrow checker will stop you), or calling mem::replace() or mem::swap(), or something along those lines.
Instead, the primary purpose of pinning is to mark these explicit relocations as unsafe for certain objects (that are referenced elsewhere by raw pointer), so that external users must promise not to relocate certain objects on pain of causing UB with your interface. In C/C++, or indeed in unsafe Rust, the same idea can be more trivially indicated by a comment such as /* Don't mess with this object until such-and-such other code is done using it! */. All pinning does is to enforce this rule at compile time for all safe code.
Interestingly, the newest Java memory feature (Panama FFI/M) actually can catch you if threads race on a memory allocation. They have done a lot of rather complex and little appreciated work to make this work in a very efficient way.
The new api lets you allocate "memory segments", which are byte arrays/C style structs. Such segments can be passed to native code easily or just used directly, deallocated with or without GC, bounds errors are blocked, use-after-free bugs are blocked, and segments can also be confined to a thread so races are also blocked (all at runtime though).
Unfortunately it only becomes available as a finalized non-preview API in Java 22, which is the release after the next one. In Java 21 it's available but behind a flag.
Memory pinning in Rust is not a problem that has to do with concurrency because the compiler will never relocate memory when something is referencing it. The problem is however with how stackless coroutines in general (even single-threaded ones, like generators) work. They are inherently self-referential structures, and Rust's memory model likes to pretend such structures don't exist, so you need library workarounds like `Pin` to work with them from safe code (and the discussion on whether they are actually sound is still open!)
>(and the discussion on whether they are actually sound is still open!)
Do you have a reference for this? Frankly, maybe I shouldn't ask since I still don't even understand why stackless coroutines are necessarily self-referential, but I am quite curious!
> I still don't even understand why stackless coroutines are necessarily self-referential, but I am quite curious!
Because when stackless coroutines run they don’t have access to the stack that existed when they were created. everything that used to be on the stack needs to get packaged up in a struct (this is what `async fn` does). However now everything that used to point to something else on the stack (which rust understands and is fine with) now points to something else within the “impl Future” struct. Hence you have self referential structs.
Basically the problem is that async blocks/fns/generators need to create a struct that holds all the local variables within them at any suspension/await/yield point. But local variables can contain references to other local variables, so there are parts of this struct that reference other parts of this struct. This creates two problems:
- once you create such self-references you can no longer move this struct. But moving a struct is safe, so you need some unsafe code that "promises" you this won't happen. `Pin` is a witness of such promise.
- in the memory model having an `&mut` reference to this struct means that it is the only way to access it. But this is no longer true for self referential structs, since there are other ways to access its contents, namely the fields corresponding to those local variables that reference other local variables. This is the problem that's still open.
> But that's nearly the opposite of what the borrow checker tries to do by statically bounding objects, at compile time.
Arc isn't an end-run around the borrow checker. If you need mutable references to the data inside of Arc, you still need to use something like a Mutex or Atomic types as appropriate.
> The degree to which a big language runtime and GC weren't a boogeyman for some problem spaces was really eye-opening.
I have the opposite experience, actually. I was an early adopter of Go and championed Garbage Collection for a long time. Then as our Go platforms scaled, we spent increasing amounts of our time playing games to appease the garbage collector, minimize allocations, and otherwise shape the code to be kind to the garbage collector.
The Go GC situation has improved continuously over the years, but it's still common to see libraries compete to reduce allocations and add complexity like pools specifically to minimize GC burden.
It was great when we were small, but as the GC became a bigger part of our performance narrative it started to feel like a burden to constantly be structuring things in a way to appease the garbage collector. With Rust it's nice to be able to handle things more explicitly and, importantly, without having to explain to newcomers to the codebase why we made a lot of decisions to appease the GC that appear unnecessarily complex at first glance.
There's a good chance this is rather a Go issue than a GC one. People get fooled by Go's pretense to be a high level C replacement. It is highly inadequate at performing this role at best.
The reason for that is the compiler quality, the design tradeoffs and Go's GC implementation throughput are simply not there for it to ever be a good general purpose systems-programming-oriented language.
Go receives undeserved hype, for use cases C# and Java are much better at due to their superior GC implementations and codegen quality (with C# offering better lower level features like structs+generics and first-class C interop).
Java GC has a non trivial overhead. I’ve moved workloads from Java to rust and gotten a 30x improvement from lack of GC. Likewise I’ve gotten 10x improvement in Java by preallocating objects and reusing then to avoid GC. (Fucking google and the cult of immutable objects). Guess what, lots of things that “make it harder to introduce bugs” make your shit run a lot slower too.
This is not an improvement from lack of GC per se but rather from zero cost abstractions (everything is monomorphised, no sin such as type erasure) first and foremost, and yes, deterministic memory management. Java is the worse language if you need to push performance to the limit since it does not offer convenient lower level language constructs to do so (unlike C#), but at reaching 80th percentile of performance, it is by far the best one.
But yes, GC is very much not free and is an explicit tradeoff vs compile time + manual memory management.
As an ops guy for decades, it makes me laugh to hear claims about Java GC superiority. Please go back in time and fix all the crashes and OOMs caused by enterprise JVM, as opposed to near-zero problems with the Go deployments.
Making stong statements without a backup in hard facts is a sign of zealotry...
I assure you if that code was to be ported to Go 1:1, Go GC would simply crawl to a halt. Write code badly enough and no matter how good hardware and software is, it won't be able to cope at some point. Even a good tool will give, if you beat it down hard enough.
Issues like these simply don't happen with GCs in modern JVM implementations or .NET (not saying they are perfect or don't have other shortcomings, but the sheer amount of developer hours invested in tuning and optimizing them far outstrips Go).
it makes me laugh to hear claims about Java GC superiority. Please go back in time and fix all the crashes and OOMs caused by enterprise JVM,
I don’t see how running into an OOM problem is necessarily a problem with the GC. That said, Java is a memory intensive language, it’s a trade off that Java is pretty up front about.
I don’t have a horse in this race but I would be quite surprised if Go’s GC implementation could even hold a candle to the ones found in C# and Java. They have spent literally decades of research and development, and god knows how much money (likely north of $1b), optimizing and refining their GC implementations. Go just simply lacks any of the sort of maturity and investment those languages have.
Java has billions spent on marketing and lobbying.
Since the advent of Java in mid-90s I hear about superiority of its VM, yet my observations from the ops PoV claim otherwise. So I suspect a huge hoax...
Hey btw, you're saying "Java is _memory intensive_", like it would magically explain everything. Let's get to that more deeply. Why is it so, dear Watson? Have you compared the memory consumption of the same algo and pretty much similar data structures between languages? Why Java has to be such a memory hog? Why also its class loading is so slow? Are these a qualities of superior VM design and zillions of man-hour invested? huh?
By the way, if the code implementing functionality X needs N times more memory than the other language with gc, then however advanced that gc would be (need to find a proof for that btw), it wouldn't catch up speedwise, because it simply needs to move around more. So simple.
Java has billions spent on marketing and lobbying.
Marketing is not a silver bullet for success and the tech industry is full of examples of exactly that. The truth is that Sun was able to promote Java so heavily because it was found to be useful.
Since the advent of Java in mid-90s I hear about superiority of its VM, yet my observations from the ops PoV claim otherwise.
The landscape of the 90s certainly made a VM language appealing. And compared to the options of that day it's hardly any wonder.
So I suspect a huge hoax...
It's you verses a plurality, if not majority, of the entire enterprise software market. Of course that's not to say that Java doesn't have problems or that the JVM is perfect, but is it so hard to believe that Java got something right? Is it honestly more believable that everyone else is caught up in a collective delusion?
Hey btw, you're saying "Java is _memory intensive_", like it would magically explain everything.
It's not that Java is necessarily memory intensive, but that a lot of Java performance tuning is focused towards optimizing throughput performance, not memory utilization. Cleaning out a large heap occasionally is in general better than cleaning out a smaller one more frequently.
By the way, if the code implementing functionality X needs N times more memory than the other language with gc, then however advanced that gc would be (need to find a proof for that btw), it wouldn't catch up speedwise, because it simply needs to move around more. So simple
It's not so simple. First of all, the choice of a large heap is not mandated by Java, it's a trade off that developers are making. Second of all, GC performance issues only manifest when code is generating a lot of garbage, and believe it or not, Java can be written to vastly minimize the garbage produced. And last of all, Java GCs like Shenandoah have a max GC pause time of less than 1ms for heaps up to 16TiB.
Anyway, at the end of the day no one is going to take Go away from you. Personally I don't have a horse in this race. That said, the fact is that Java GCs are far more configurable, sophisticated, and advanced than anything Go has (and likely ever will). IMO, Go came at a point in time where there was a niche to exploit, but that niche is shrinking.
I would like to answer your points more deeply, not having much time for it now.
But I think you are avoiding a direct answer to the question why Java needs so much memory in the first place. You say about "developer's choice for a big heap", first I don't think it is their choice, but the consequence of the fact that such a big heap is needed at all, for a typical code. Why?
Let's code a basic https endpoint using typical popular framework returning some simple json data. Usually stuff. Why it will be consuming 5x - 10x more memory for Java? And, if one says it's just unrealistic microbenchmark, things go worse when coding more real stuff.
Btw,having more knobs for a gc is not necessarily a good thing, if it means that there are no fire-and-forget good defaults. If an engineer needs continously to get his head around these knobs to have a non-crashing app, then we have problem. Or rather - ops have a problem, and some programmers are, unfortunately, disconnected from the ops realm. Have you been working together with ops guys? On prod, ofc?
> In the same sense that the lifetime of an object in a GC'd system has a lower bound of, "as long as it's referenced", sure.
These are not the same.
The problem with GC'd systems is that you don't know when the GC will run and eat up your cpu cycles. It is impossible to determine when the memory will actually be freed in such systems. With ARC, you know exactly when you will release your last reference and that's when the resource is freed up.
In terms of performance, ARC offers massive benefits because the memory that's being dereferenced is already in the cache. It's hard to understate how big of a deal this is. There's a reason people like ARC and stay away from GC when performance actually begins to matter. :)
> With ARC, you know exactly when you will release your last reference and that's when the resource is freed up.
It's more like "you notice when it happens". You don't know in advance when the last reference will be released (if you did, there would be no point in using reference counting).
> In terms of performance, ARC offers massive benefits because the memory that's being dereferenced is already in the cache.
It all depends on your access patterns. When ARC adjusts the reference counter, the object is invalidated in all other threads' caches. If this happens with high frequency, the cache misses absolutely demolish performance. GC simply does not have this problem.
> There's a reason people like ARC and stay away from GC when performance actually begins to matter.
If you're using a language without GC built in, you usually don't have a choice. When performance really begins to matter, people reach for things like hazard pointers.
> It's more like "you notice when it happens". You don't know in advance when the last reference will be released
A barista knows when a customer will pay for coffee (after they have placed their order). A barista does not know when that customer will walk in through the door.
> (if you did, there would be no point in using reference counting).
There’s a difference between being able to deduce when the last reference is dropped (for example, by profiling code) and not being able to tell anything about when something will happen.
A particular developer may not know when the last reference to an object is dropped, but they can find out. Nobody can guess when GC will come and take your cycles away.
> The cache misses absolutely demolish performance
With safe Rust, you shouldn’t be able to access memory that has been freed up. So cache misses on memory that has been released is not a problem in a language that prevents use-after-free bugs :)
> If you’re using a language without GC built in, you usually don’t have a choice.
I’m pretty sure the choice of using Rust was made precisely because GC isn’t a thing (in all places that love and use rust that is)
> A barista knows when a customer will pay for coffee (after they have placed their order). A barista does not know when that customer will walk in through the door.
Sorry, no chance of deciphering that.
> There’s a difference between being able to deduce when the last reference is dropped (for example, by profiling code) and not being able to tell anything about when something will happen.
> A particular developer may not know when the last reference to an object is dropped, but they can find out.
The developer can figure out when the last reference to the object is dropped in that particular execution of the program, but not in the general sense, not anymore than they can in a GC'd language.
The only instance where they can point to a place in the code and with certainty say "the reference counted object that was created over there is always destroyed at this line" is in cases where reference counting was not needed in the first place.
> With safe Rust, you shouldn’t be able to access memory that has been freed up. So cache misses on memory that has been released is not a problem in a language that prevents use-after-free bugs :)
I'm not sure why you're talking about freed memory.
Say that thread A is looking at a reference-counted object. Thread B looks at the same object, and modifies the object's reference counter as part of doing this (to ensure that the object stays alive). By doing so, thread B has invalidated thread A's cache. Thread A has to spend time reloading its cache line the next time it accesses the object.
This is a performance issue that's inherent to reference counting.
> I’m pretty sure the choice of using Rust was made precisely because GC isn’t a thing (in all places that love and use rust that is)
Wanting to avoid "GC everywhere", yes. But Rust/C++ programs can have parts that would be better served by (tracing) garbage collection, but where they have to make do with reference counting, because garbage collection is not available.
> In terms of performance, ARC offers massive benefits
but it also has big disadvantage, that it communicates to actual malloc for memory management, which is usually much less performant than GC from various reasons.
> which is usually much less performant than GC from various reasons.
Can you elaborate?
I've seen a couple of malloc implementations, and in all of them, free() is a cheap operation. It usually involves setting a bit somewhere and potentially merging with an adjacent free block if available/appropriate.
malloc() is the expensive call, but I don't see how a GC system can get around the same costs for similar reasons.
I've seen some benchmarks, but can't find them now, so maybe I am wrong about this.
> free() is a cheap operation. It usually involves setting a bit somewhere and potentially merging with an adjacent free block if available/appropriate.
there is some tree like structure somewhere, which then would allow to locate this block for "malloc()", this structure has to be modified in parallel by many concurrent threads, which likely will need some locks, meaning program operates outside of CPU cache.
In JVM for example, GC is integrated into thread models, so they can have heap per thread, and also "free()" happens asynchronously, so doesn't block calling code. Additionally, malloc approaches usually suffer from memory fragmentation, while JVM GC is doing compactions all the time in background, tracks memory blocks generations, and many other optimizations.
- Like others have said, both malloc()/free() touch a lot of global state, so you either have contention between threads, or do as jemalloc does and keep thread-local pools that you occasionally reconcile.
- A moving (and ideally, generational) GC means that you can recompact the heap, making malloc() little more than a pointer bump.
- This also suggests subsequent allocations will have good locality, helping cache performance.
Manual memory management isn't magically pause-free, you just get to express some opinion about where you take the pauses. And I'll contend that (A) most programmers aren't especially good at choosing when that should be, and (B) lots (most?) software cares about overall throughput, so long as max latency stays under some sane bound.
GC generally optimises for throughput over latency. But there is also another cost: high-throughput GC usually uses more memory (sometimes 2-3x as much!). Arc keeps your memory usage low and can keep your latency more consistent, but it will often sacrifice throughput compared to a GC tuned for it. (Of course, stack allocation, where possible, beats them all, which is why rust and C++ tend to win out over java in throughput even if the GC has an advantage over reference counting, because java has to GC a lot more than other languages due to no explicit stack allocation)
This feels like a large portion of the criticism. I thought this article was going to be more about how the async transformation gets in the way of a lot of transformations that the compiler could make in non-async code.
The point about wrangling with Weak suggests that they're trying to build complex ownership structures (which, to be fair, would be easier in to deal with a single thread) which isn't really something easy to express in Rust in general. I use weak smart pointers exceedingly rarely. Outside of the first section (which isn't talking about async Rust specifically, it's just speaking about concurrency generally) channels aren't even mentioned. They're the main thing I use for communication between different parts of my program when writing async code and when interfacing between async and non-async code, plus the other signalling abstractions like Notify, semaphores, etc. Mutexes are slow and bottlenecky and shared state quickly gets complicated to manage, this has been known for ages. I think the problem might be more the `BIG_GLOBAL_STATIC_REF_OR_SIMILAR_HORROR` in the first place.
The comment about nothing stopping you from calling blocking code in an async context is valid, but it's relatively manageable and you can use `tokio::spawn_blocking` or similar when you must do it.
Reference counting is a type of GC [0]. Just not a very good one in many cases.
I think it's a fair assumption to say that the author is aware of what Arcs are and how they work. I believe their point is more so that because of how async works in Rust, users have to reach for Arc over normal RAII far more often than in sync code. So at a certain point, if you have a program where 90% of objects are refcounted, you might as well use a tracing GC and not have the overhead of many small heap allocations/frees plus atomic ops.
Perhaps there are in fact ways around Arc-ing things for the author's use cases. But in my (limited) experience with Rust async I've definitely run into things like this, and plenty of example code out there seems to do the same thing [1].
For what it's worth, I've definitely wondered whether a real tracing GC (e.g. [2]) could meaningfully speed up many common async applications like HTTP servers. I'd assume that other async use cases like embedded state machines would likely have pretty different performance characteristics, though.
> Reference counting is a type of GC [0]. Just not a very good one in many cases.
…on a server where you can have a ton of RAM. It's superior on client machines because it's friendlier to swapped out memory, which is why Swift doesn't have a GC.
I love Rust, but async is a hot mess and you cannot just write async code the same way that you write sync code. I'm getting more convinced that mixing the two is a bad idea, and that Go's approach of making everything sync with a single async channel primitive might be right.
I'm currently plumbing through some logic to call a sync method on a struct that implements Future and it's... an interesting challenge.
While we can make zero-cost async abstractions somewhat easy for users, the library developers are the ones who suffer the pain.
Library developers can afford to deal with complexity much more than users of libraries. Offloading such work on highly skilled people developing the basic infrastructure is surely the right approach.
I totally agree that library developers are the ones who _can_ handle complexity, but I have found even some of the top Rust devs are making async mistakes -- either the APIs are not correct from an async perspective, or there are basic bugs like losing wakers. The latter is so common it's not funny.
I've seen one wasm VM for Rust that offered what looked like transparent M:N, which should solve (in that case) most async difficulties. We'll see how that evolves.
A good candidate for this is Graal. It can compile (JIT/AOT) both WASM and also LLVM bitcode directly so Rust programs can have full hardware/OS access without WASM limitations, and in theory it could allow apps to fully benefit from the work done on Loom and async. The pieces are all there. The main issue is you need to virtualize IO so that it goes back into the JVM, so the JVM controls all the code on the stack at all times. I think Graal can do this but only in the enterprise edition. Then you'd be able to run ~millions of Rust threads.
I disagree with you in the last point, async is definitely painful for end users. It indeed feels like you're using a completely different language, which has Rust's core features removed – lifetimes and explicit types, sprinkled with a mess of Pins on top.
You cannot run scoped fibers, forcing you to "Arc shit up", Pins are unusable without unsafe, and a tiniest change in an async-function could make the future !Send across the entire codebase.
This is bound to get some criticism (or some tangent-at-best discussion), but it seems like a pretty fair discussion to me.
What I'm missing at the end of the article is the author's point: I believe they're advocating for the use of raw threads and manual management of concurrency, and doing away with the async paraphernalia. But, at the same time, earlier in the article they give the example of networking-related tasks as something that isn't so easy to deal with using only raw threads.
So, taking into account that await&co. are basically syntactic sugar + an API standard (iirc, I haven't used Rust so much lately), I wonder about what the alternative is. In particular, it seems to me like the alternative you could have would be everyone rolling their own "concurrency API", where each crate (inconsistently) exposes some sort of `await()` function, and you have to manually roll your async runtime every time. This would obviously also not be ideal.
I thought the author's point was relatively clear: Rust might not be a good fit for the kind of tasks that need more concurrency than raw threads can give you. Such programs should be written in some other language instead.
> Maybe Rust isn’t a good tool for massively concurrent, userspace software. We can save it for the 99% of our projects that don’t have to be.
Anything that waits on I/O needs concurrency (but not necessarily threads). Web backends, web frontends, deeper backends, desktop GUIs, that's probably 90% of software right there.
I interpreted the 99% thing as referring to all software. If it's just Rust projects then sure, then again anyone who needs async has probably been avoiding a language that lacked async until recently.
The author's point is that Rust is not a good language for software like that example. But very, very little software is like that, and you can always divide it up in large blocks inside of what Rust fits quite well.
Personally, I'm a bit more radical than the author. You won't be able to write software like the example correctly. It should just not be done, ever. Machines can still optimize some sanely organized software into the same thing, maybe, if it happens to be a tractable problem (I'm not sure anybody knows). But people shouldn't touch that thing.
Now, I think async is bad syntactic sugar that hides what's really going on under the surface. And I rail against it all the time. Especially the way dropping in async contaminates code bases by building tendrils across call-sites all through the application. ... But the tools that have been built around it are very useful and there's some good stuff there.
I have some quibbles with this article:
"Rust comes at this problem with an “async/await” model"
No, it does not. It allows for that, and there's a big ... community ... around the async stuff, but in reality the language is entirely fine with operating using explicit concurrency constructs. And in fact for most applications I think standard synchronous functions combined with communicating channels is cleaner. I work in code bases that do both, and I find the explicit approach easier to reason about.
In the end, Async is something people ideally reach for only after they hit the wall with blocking on I/O. But in reality they're often reaching for it just because -- either because it's cool... or because some framework they are relying on mandates it.
But I think the pendulum will swing back the other way at some point. I don't think it's fair to tar the whole language with it.
Practically speaking: the only libraries I regularly use that demand me to use async stuff are HTTP clients, and those have optional blocking imports. I still need to sprinkle [tokio::main] in front of the entry point, but from that point on, everything is blocking.
I don't think it's "the entire community" at all. Dealing with futures across library calls is a pain and almost every library that can avoid it, will avoid it.
I try to avoid async code because of its annoying pain points and I rarely see any circumstances where spawning a new thread doesn't work. Sure, there's more overhead, and you need some kind of limiting factor to prevent spawning a billion of them, but async isn't really required in most circumstances.
It's like saying Go allows for generics. Very few people and libraries bother with them. Working with them is kind of a pain. They're there jf you want to use them, but you generally don't.
You are wrong. Very wrong. Async Rust is the best. Best Rust is async. I’m going to await at you. Async rust is good. Rust is fast and safe. More than C.
I find myself in this weird corner when it comes to async rust.
The guy's got a point in that doing a bunch of Arc, RwLock, and general sharing of state is going to get messy. Especially once you are sprinkling 'static all over the place, it infects everything, much like colored functions. I did this whole thing once back when I was starting off where I would Arc<RwLock> stuff, and try to be smart about borrow lifetimes. Total mess.
But then rust also has channels. When you read about it, it talks about "messages", which to me means little objects. Like a few bytes little. This is the solution, pretty much everything I write now is just a few tasks that service some channels. They look at what's arrived and if there's something to output, they will put a message on the appropriate channel for another task to deal with. No sharing objects or anything. If there's a large object that more than one task needs, either you put it in a task that sends messages containing the relevant query result, or you let each task construct its own copy from the stream of messages.
And yet I see a heck of a lot of articles about how to Arc or what to do about lifetimes. They seem to be things that the language needs, especially if you are implementing the async runtime, but I don't understand why the average library user needs to focus so much on this.
But green threads were not and are not the right solution for Rust, so it's kind of beside the point. Async Rust is difficult, but it will eventually be possible to use Async Rust inside the Linux kernel, which is something you can't do with the Go approach.
Maybe "add a runtime that switches execution contexts on behalf of the user" and "force the programmer to reimplement everything" are not the only options.
We need a way to bridge the gap. Having a runtime may not be suitable for all apps but it can easily allow you to reach 95%+ concurrency performance. The async compile-to-state-machine model is only necessary for the last 5%. Most userland apps rarely need to maximize concurrency efficiency. They need concurrency yes, but performance at the 95th percentile is more than sufficient.
I really don’t buy this argument that only some small “special” fraction of apps “actually” need async, and for the rest of us “plebs” we should be relegated to blocking.
Async is just hard. That’s it. It’s fundamentally difficult.
In my experience language implementations of async fall into 2-axes: clarity and control. C# is straightforward-enough (having cribbed its async design off functional languages) but I find it scores low on the “clarity” scale and moderate-high in control, because you could control it, but it was t always clear.
JS is moderate-high clarity, low control: easy to understand, because all the knobs are set for you. Before it got async/await sugar, I’d have said it would have been low clarity, because I’ve seen the promise/callback hell people wrote when given rope.
Python is the bottom of the barrel for both clarity and control. It genuinely has to have the most awful and confusing async design I’ve ever seen.
I personally find Rust scores high in both clarity and control. Playing with the Glommio executor was what really solidified my understanding of how async works however.
I learned concurrency and parallelism by confronting blocking behavior: waiting on a networking or filesystem request stops the world, so we need a new execution context to keep things moving.
What I realized, eventually, is that blocking is a beautiful thing. Embrace the thread of execution going to sleep, as another thread may now execute on the (single core at the time) CPU.
Now you have an organization problem, how to distribute threads across different tasks, some sequential, some parallel, some blocking, some nonblocking. Thread-per-request? Thread-per-connection?
And now a management problem. Spawning threads. Killing threads. Thread pools. Multithreaded logging. Exceptions and error handling.
Totally manageable in mild cases, and big wins in throughput, but scaling limits will present themselves.
I confront many of these tradeoffs in a fun little exercise I call "Miner Mover", implemented in Ruby using many different concurrency primitives here: https://github.com/rickhull/miner_mover
Rust Futures are essentially green threads, except much lighter-weight, much faster, and implemented in user space instead of being built-in to the language.
Basically Rust Futures is what Go wishes it could have. Rust made the right choice in waiting and spending the time to design async right.
You're overstating your case. Rust's async tasks (based on stackless coroutines) and Go's goroutines (based on stackful coroutines) have important differences. Rust's design introduces function coloring (tentative solution in progress) but is much more suited for the bare-metal scene that C and C++ are famous for. Go's design has more overhead but, by virtue of not having colored functions, is simpler for programmers to write code for. Most things in computer science/programming involve tradeoffs. Also, Rust's async/await is built-in to the language. It's not a library implementation of stackless coroutines.
> Go's design has more overhead but, by virtue of not having colored functions, is simpler for programmers to write code for.
Colored functions is a debatable problem at best. I consider it a feature not a bug and it makes reasoning about programs easier at the expense of writing additional async/await keywords which is really a very minor annoyance.
On the other hand Go's need of using channels to do trivial and common tasks like communicating the result of an async task together with the lack of RAII and proper cleanup signaling in channels (you can very easily deadlock if nothing is attached on the other end of the channel), plus no compile time race detection - all that makes writing concurrent code harder.
If Rust had gone with green threads as the core async strategy (I know it was a thing pre-1.0), that would be terrible. You're not understanding Rust's design goals. Rust's async model, while it has several major pain points at present, is still undoubtedly superior for what Rust was made for. It would be a shame to throw all that away. Go can go do it's own thing (it has, evidently).
It really is, but I still favour "unsexy" manual poll/select code with a lot of if/elseing if it means not having to deal with async.
I fully acknowledge that I'm an "old school" system dev who's coming from the C world and not the JS world, so I probably have a certain bias because of that, but I genuinely can't understand how anybody could look at the mess that's Rust's async and think that it was a good design for a language that already had the reputation of being very complicated to write.
I tried to get it, I really did, but my god what a massive mess that is. And it contaminates everything it touches, too. I really love Rust and I do most of my coding in it these days, but every time I encounter async-heavy Rust code my jaw clenches and my vision blurs.
At least my clunky select "runtime" code can be safely contained in a couple functions while the rest of the code remains blissfully unaware of the magic going on under the hood.
Dear people coming from the JS world: give system threads and channels a try. I swear that a lot of the time it's vastly simpler and more elegant. There are very, very few practical problems where async is clearly superior (although plenty where it's arguably superior).
I think Rust’s async stuff is a little half baked now but I have hope that it will be improved as time goes on.
In the mean time it is a little annoying to use, but I don’t mind designing against it by default. I feel less architecturally constrained if more syntactically constrained.
I'm curious what things you consider to be half-baked about Rust async.
I've used Rust async extensively for years, and I consider it to be the cleanest and most well designed async system out of any language (and yes, I have used many languages besides Rust).
Async traits come to mind immediately, generally needing more capability to existentially quantify Future types without penalty. Async function types are a mess to write out. More control over heap allocations in async/await futures (we currently have to Box/Pin more often than necessary). Async drop. Better cancellation. Async iteration.
> That would be useful, but I wouldn't call the lack of it "half-baked", since no other mainstream language has it either. It's just a nice-to-have.
Golang supports running asynchronous code in defers, similar with Zig when it still had async.
Async-drop gets upgraded from a nice-to-have into an efficiency concern as the current scheme of "finish your cancellation in Drop" doesn't support borrowed memory in completion-based APIs like Windows IOCP, Linux io_uring, etc. You have to resort to managed/owned memory to make it work in safe Rust which adds unnecessary inefficiency. The other alternatives are blocking in Drop or some language feature to statically guarantee a Future isn't cancelled once started/initially polled.
To run async in Drop in rust, you need to use block_on() as you can't natively await (unlike in Go). This is the "blocking on Drop" mentioned and can result in deadlocks if the async logic is waiting on the runtime to advance, but the block_on() is preventing the runtime thread from advancing. Something like `async fn drop(&mut self)` is one way to avoid this if Rust supported it.
You need to `block_on` only if you need to block on async code. But you don't need to block on order to run async code. You can spawn async code without blocking just fine and there is no risk of deadlocks.
Ok, in a very, very rare case (so far never happened to me) when I really need to await an async operation in the destructor, I just define an additional async destructor, call it explicitly and await it. Maybe it's not very elegant but gets the job done and is quite readable as well.
And this would be basically what you have to do in Go anyways - you need to explicitly use defer if you want code to run on destruction, with the caveat that in Go nothing stops you from forgetting to call it, when in Rust I can at least have a deterministic guard that would panic if I forget to call the explicit destructor before the object getting out of scope.
BTW async drop is being worked on in Rust, so in the future this minor annoyance will be gone
Yes I am aware of async drop proposals. And the point is not to handle a single value being dropped but to facilitate invariants during an abrupt tear down. Today, when I am writing a task which needs tear down I need to hand it a way to signal a “nice” shutdown, wait some time, and then hard abort it.
1) That's no longer "running async code in Drop" as it's spawned/detached and semantically/can run outside the Drop. This distinction is important for something like `select` which assumes all cancellation finishes in Drop. 2) This doesn't address the efficiency concern of using borrowed memory in the Future. You have to either reference count or own the memory used by the Future for the "spawn in Drop" scheme to work for cleanup. 3) Even if you define an explicit/custom async destructor, Rust doesn't have a way to syntactically defer its execution like Go and Zig do so you'd end up having to call it on all exit points which is error prone like C (would result in a panic instead of a leak/UB, but that can be equally undesirable). 4) Is there anywhere one can read up on the work being done for async Drop in Rust? Was only able to find this official link but it seems to still have some unanswered questions (https://rust-lang.github.io/async-fundamentals-initiative/ro...)
Re: existential quantification and async function types
It'd be very nice to be able to use `impl` in more locations, representing a type which needs not be known to the user but is constant. This is a common occurrence and may let us write code like `fn foo(f: impl Fn() -> impl Future)` or maybe even eventually syntax sugar like `fn foo(f: impl async Fn())` which would be ideal.
Re: Boxing
I find that a common technique needed to get make abstraction around futures to work is the need to Box::pin things regularly. This isn't always an issue, but it's frequent enough that it's annoying. Moreover, it's not strictly necessary given knowledge of the future type, it's again more of a matter of Rust's minimal existential types.
Re: async drop and cancellation.
It's not always possible to have good guarantees about the cleanup of resources in async contexts. You can use abort, but that will just cause the the next yield point to not return and then the Drops to run. So now you're reliant on Drops working. I usually build in a "kind" shutdown with a timer before aborting in light of this.
C# has a version of this with their CancelationTokens. They're possible to get wrong and it's easy to fail to cancel promptly, but by convention it's also easy to pass a cancelation request and let tasks do resource cleanup before dying.
Re: Async iteration
Nicer syntax is definitely the thing. Futures without async/await also could just be done with combinators, but at the same time it wasn't popular or easy until the syntax was in place. I think there's a lot of leverage in getting good syntax and exploring the space of streams more fully.
> It really is, but I still favour "unsexy" manual poll/select code with a lot of if/elseing if it means not having to deal with async.
> I fully acknowledge that I'm an "old school" system dev who's coming from the C world and not the JS world, so I probably have a certain bias because of that, but I genuinely can't understand how anybody could look at the mess that's Rust's async and think that it was a good design for a language that already had the reputation of being very complicated to write.
I'm in the same "old school" system dev category as you, and I think that modern languages have gone off the deep end, and I complained about async specifically in a recent comment on HN: https://news.ycombinator.com/item?id=37342711
> At least my clunky select "runtime" code can be safely contained in a couple functions while the rest of the code remains blissfully unaware of the magic going on under the hood.
And we could have had that for async as well, if languages were designed by the in-the-trenches industry developer, and not the "I think Haskell and Ocaml is great readability" academic crowd.
With async in particular, the most common implementation is to color the functions by qualifying the specific function as async, which IMO is exactly the wrong way to do it.
The correct way would be for the caller to mark a specific call as async.
IOW, which of the following is clearer to the reader at the point where `foo` is called?
Option 1: color the function
async function foo () {
// ...
}
...
let promise = foo ();
let bar = await promise;
Option 2: schedule any function
function foo () {
// ...
}
let sched_id = schedule foo ();
...
let bar = await sched_id;
Option 1 results in compilation errors for code in the call-stack that isn't async, results in needing two different functions (a wrapper for sync execution), and means that async only works for that specific function. Option 2 is more like how humans think - schedule this for later execution, when I'm done with my current job I'll wait for you if you haven't finished.
Isn't mixing async and sync code like this a recipe for deadlocks?
What if your example code is holding onto a thread that foo() is waiting to use?
Said another way, explain how you solved the problems of just synchronously waiting for async. If that just worked then we wouldn't need to proliferate the async/await through the stack.
> Said another way, explain how you solved the problems of just synchronously waiting for async.
Why? It isn't solved for async functions, is it? Just because the async is propagated up the call-stack doesn't mean that the call can't deadlock, does it?
Deadlocks aren't solved for a purely synchronous callstack either - A grabbing a resource, then calling B which calls C which calls A ...
Deadlocks are potentially there whether or not you mix sync/async. All that colored functions will get you is the ability to ignore the deadlock because that entire call-stack is stuck.
> If that just worked then we wouldn't need to proliferate the async/await through the stack.
> Yes actually it is solved. If you stick to async then it cannot deadlock (in this way) because you yield execution to await.
Maybe I'm misunderstanding what you are saying. I use the word "_implementation_type_" below to mean "either implemented as option 1 or option 2 from my post above."
With current asynchronous implementations (like JS, Rust, etc), any time you use `await` or similar, that statement may never return due to a deadlock in the callstack (A is awaiting B which is awaiting C which is awaiting A).
And if you never `await`, then deadlocking is irrelevant to the _implementation_type_ anyway.
So I am trying to understand what you mean by "it cannot deadlock in this way" - in what way do you mean? async functions can accidentally await on each other without knowing it, which is the deadlock I am talking about.
I think I might understand better if you gave me an example call-chain that, in option 1, sidesteps the deadlock, and in option 2, deadlocks.
I'm referring to the situation where a synchronous wait consumes the thread pool, preventing any further work.
A is sychrounously waiting B which is awaiting C which could complete but never gets scheduled because A is holding onto the only thread. Its a very common situation when you mix sync and async and you're working in a single threaded context, like UI programming with async. Of course it can also cause starvation and deadlock in a multithreaded context as well but the single thread makes the pitfall obvious.
That's an implementation problem, not a problem with the concept of asynchronous execution, and it's specifically a problem in only one popular implementation: Javascript in the browser without web-workers.
That's specifically why I called it a Leaky Abstraction in my first post on this: too many people are confusing a particular implementation of asynchronous function calls with the concept of asynchronous function calls.
I'm complaining about how the mainstream languages have implemented async function calls, and how poorly they have done so. Pointing out problems with their implementation doesn't make me rethink my position.
I don't see how it can be an implementation detail when fundamentally you must yield execution when the programmer has asked to retain execution.
Besides Javascript, its also a common problem in C# when you force synchronous execution of an async Task. I'm fairly sure its a problem in any language that would allow an async call to wait for a thread that could be waiting for it.
I really can't imagine how your proposed syntax could work unless the synchronous calls could be pre-empted, in which case, why even have async/await at all?
> I don't see how it can be an implementation detail when fundamentally you must yield execution when the programmer has asked to retain execution.
It's an implementation issue, because "running on only a single thread" is an artificial constraint imposed by the implementation. There is nothing in the concept of async functions, coroutines, etc that has the constraint "must run on the same thread as the sync waiting call".
An "abstraction" isn't really one when it requires knowledge of a particular implementation. Async in JS, Rust, C#, etc all require that the programmer knows how many threads are running at a given time (namely, you need to know that there is only one thread).
> But I look forward to your implementation.
Thank you :-)[1]. I actually am working (when I get the time, here and there) on a language for grug-brained developers like myself.
One implementation of "async without colored functions" I am considering is simply executing all async calls for a particular host thread on a separate dedicated thread that only ever schedules async functions for that host thread. This sidesteps your issue and makes colored functions pointless.
This is one possible way to sidestep the specific example deadlock you brought up. There's probably more.
[1] I'm working on a charitable interpretation of your words, i.e. you really would look forward to an implementation that sidesteps the issues I am whining about.
> and not the "I think Haskell and Ocaml is great readability" academic crowd.
Actually, Rust could still learn a lot from these languages. In Haskell, one declares the call site as async, rather than the function. OCaml 5 effect handlers would be an especially good fit for Rust and solve the "colouration" problem.
> but I genuinely can't understand how anybody could look at the mess that's Rust's async and think that it was a good design for a language that already had the reputation of being very complicated to write.
Rust adopted the stackless coroutine model for async tasks based on its constraints, such as having a minimal runtime by default, not requiring heap allocations left and right, and being amenable to aggressive optimizations such as inlining. The function coloring problem ("contamination") is an unfortunate consequence. The Rust devs are currently working on an effects system to fix this. Missing features such as standard async traits, async functions in traits, and executor-agnosticism are also valid complaints. Considering Rust's strict backwards compatibility guarantee, some of these will take a long time.
I like to think of Rust's "async story" as a good analogue to Rust's "story" in general. The Rust devs work hard to deliver backwards compatible, efficient, performant features at the cost of programmer comfort (ballooning complexity, edge cases that don't compile, etc.) and compile time, mainly. Of course, they try to resolve the regressions too, but there's only so much that can be done after the fact. Those are just the tradeoffs the Rust language embodies, and at this point I don't expect anything more or less. I like Rust too, but there are many reasons others may not. The still-developing ecosystem is a prominent one.
I read comments like this and feel like I’m living in some weird parallel universe. The vast majority of Rust I write day in and day out for my job is in an async context. It has some rough edges, but it’s not particularly painful and is often pleasant enough. Certainly better than promises in JS. I have also used system threads, channels, etc., and indeed there are some places where we communicate between long running async tasks with channels, which is nice, and some very simple CLI apps and stuff where we just use system threads rather than pulling in tokio and all that.
Anyway, while I have some issues with async around futur composition and closures, I see people with the kind of super strong reaction here and just feel like I must not be seeing something. To me, it solves the job well, is comprehensible and relatively easy to work with, and remains performant at scale without too much fiddling.
Honestly, this is me too. The only thing I’d like to also see is OTP-like supervisors and Trio-like nurseries. They each have their use and they’re totally user land concerns.
Actually, this "old school" approach is more readable even for folks who have never worked in the low-level C world. At-least everything is in front of your eyes and you can follow the logic. Unless code leveraging async is very well-structured, it requires too much brain-power to process and understand.
Exactly. People are too afraid of using threads these days for some perceived cargo-cult scalability reasons. My rule of thumb is just to use threads if the total number of threads per process won't exceed 1000.
(This is assuming you are already switching to communicating using channels or similar abstraction.)
Async and GUI threads are different concepts. Of course most GUIs have an event loop which can be used as a form of async, but with async you do your calculations in the main thread, while with GUIs you typically spin your calculations off to a different thread.
Most often when doing async you have a small number of tasks repeated many times, then you spin up one thread per CPU, and "randomly" assign each task as it comes in to a thread.
When doing GUI style programming you have a lot of different tasks and each task is done in exactly one thread.
Hmm I would say the concepts are intertwined. Lots of GUI frameworks use async/await and the GUI thread is just another concurrency pattern that adds lock free thread exclusivity to async tasks that are pinned to a single thread.
Note that if you "just" write responses to queries without yielding execution, you don't need async, you just write Sync handlers to an async framework. (Hitting dB requests in a synchronous way is not good for your perf though, you better have a mostly read / well cached problem)
The challenge is that async colors functions and many of the popular crates will force you to be async, so it isn't always a choice depending on which crates you need.
Please excuse my ignorance, I haven't done a ton of async Rust programming - but if you're trying to call async Rust from sync Rust, can you not just create a task, have that task push a value through a mpsc channel, shove the task on the executor, and wait for the value to be returned? Is the concern that control over the execution of the task is too coarse grained?
There are ways to call both from both for sure, but my point is if you don't want any async in your code at all...that often isn't a choice if you want to use the popular web frameworks for example.
Yes, you can do that. You can use `block_on` to convert an async Future into a synchronous blocking call. So it is entirely possible to convert from the async world back into the sync world.
But you have to pull in an async runtime to do it. So library authors either have to force everyone to pull in an async runtime or write two versions of their code (sync and async).
The performance overhead of threads is largely unrelated to how many you have. The thing being minimized with async code is the rate at which you switch between them, because those context switches are expensive. On modern systems there are many common cases where the CPU time required to do the work between a pair of potentially blocking calls is much less than the CPU time required to yield when a blocking call occurs. Consequently, most of your CPU time is spent yielding to another thread. In good async designs, almost no CPU time is spent yielding. Channels will help batch up communication but you still have to context switch to read those channels. This is where thread-per-core software architectures came from; they use channels but they never context switch.
Any software that does a lot of fine-grained concurrent I/O has this issue. Database engines have been fighting this for many years, since they can pervasively block both on I/O and locking for data model concurrency control.
The cost of context switching in "async" code is very rarely smaller than the cost of switching OS threads. (Exception is when you'ree using a GC language with some sort of global lock.)
"Async" in native code is cargo cult, unless you're trying to run on bare metal without OS support.
Nodejs is inherently asynchronous and the JavaScript developers bragged during its peak years how it was faster than Java for webservers despite only using one core because a classic JEE servlet container launches a new thread per request. Even if you don't count this as "context switch" and go for a thread pool you are deluding yourself because a thread pool is applying the principles of async with the caveat that tasks you send to the thread pool are not allowed to create tasks of their own.
There is a reason why so many developers have chosen to do application level scheduling: No operating system has exposed viable async primitives to build this on the OS level. OS threads suck so everyone reinvents the wheel. See Java's "virtual threads", Go's goroutines, Erlang's processes, NodeJS async.
You don't seem to be aware what a context switch on an application level is. It is often as simple as a function call. There is no way that returning to the OS, running a generic scheduler that is supposed to deal with any possible application workload that needs to store all the registers and possibly flush the TLB if the OS makes the mistake of executing a different process first and then restore all the registers can be faster than simply calling the next function in the same address space.
Developers of these systems brag about how you can have millions of tasks active at the same time without breaking any sweat.
The cost of switching goroutines, rust Futures, Zig async Frames, or fibers/userspace-tasks in general is on the other of a few nano-seconds whereas it's in the micro-second range for OS threads. This allows you to spawn tons of tasks and have them all communicate with each other very quickly (write to queue; push receiver to scheduler runqueue; switch out sender; switch to receiver) whereas doing so with OS threads would never scale (write to queue; syscall to wake receiver; syscall to switch out sender). Any highly concurrent application (think games, simulations, net services) uses userspace/custom task scheduling for similar reasons.
I can't both perform blocking I/O and wait for a cancellation signal from another thread. So I need to use poll(), and async is a nice interface to that.
A particularly interesting use case for async Rust without threads is cooperative scheduling on microcontrollers[1]; this article also does a really good job of explaining some of the complications referenced in TFA.
You cancel a sync IO op similar to how you cancel an async one: have another task (i.e OS thread in this case) issue the cancellation. Select semantically spawns a task per case/variant and does something similar under the hood if cancellation is implemented.
You can do that, but then the logic of your cancellable thread gets intermingled with the cancellation logic.
And since the cancellation logic runs on the cancellable thread, you can't really cancel a blocking operation. What you can do is to let it run to completion, check that it was canceled, and discard the value.
Not sure I follow; the cancellation logic is on both threads/tasks 1) the operation itself waiting for either the result or a cancel notification and 2) the cancellation thread sending that notification.
The cancellation thread is generally the one doing the `select` so it spawns the operation thread(s) and waits for (one of) their results (i.e. through a channel/event). The others which lose the race are sent the cancellation signal and optionally joined if they need to be (i.e. they use intrusive memory).
He didn't say queues though. CSP isn't processes streaming data to each other through buffered channels, it's one process synchronously passing one message to another. Whichever one gets the the communication point waits for the other.
I really like the message passing paradigm. And languages like Erlang have shown that its an excellent choice... for distributed systems. But writing code like that is a very diffferent experience from, say, async JavaScript, which feels more like writing synchronous code with green threads (except you have to deal with function coloring as well). I believe people will try to write code in a way that is already familiar to them, leading them down the path of Arc and RwLock in Rust.
> I imagine Rust to be a language far more similar to Go, in both use cases and functionality, than JS.
I mostly agree. But I would wager that for a significant amount of people their first exposure to "async" is JS and not any number of other languages. And when you try to write async Rust the same way as you might write async JS, things just aren't that pretty.
And in the end, almost everything ends up using Mutex, RWMutex, WaitGroup, Once, and some channels that exist only to ever be closed (like Context.Done), and only if you need to select around them.
As a quite senior Go developer, I'd like to +1 this a ton. You're far more likely to have shocking edge cases unaccounted for when using channels. I consider every usage very, very carefully. Just like every other language, I think the ultimate solution is to build higher-level abstractions for concurrency patterns (e.g. errgroup) and, now that Go has generics, it's the right time to start building them.
The first one is indeed non-obvious, but the remaining snippets presented as bugs would not pass a review unless hidden inside 1k+ LOC PRs. Some are so blatantly obvious (seriously for loop and not passing current value as variable?) that I'm surprised that authors have listed them as if they're somehow special.
> for loop and not passing current value as variable
In most languages, current for loop value is always accessed a variable, not a reference. The only languages where it's not the case that I know of are Go and Python (JavaScript used to also have this problem with for(var ...), it was fixed with for(let ...)). So if you don't regularly write Go, it's easy to make this mistake.
I still like channels because they may be a net reduction in the number of concurrency primitives in use, which complicates quantification in the paper - their taxonomy is great, though. Channels have some sharp corners.
Reducing the number of concurrency primitives does not imply reduction in complexity. On the contrary in fact, I've seen the messes created by golang in large production systems. Here's a good article: https://www.uber.com/blog/data-race-patterns-in-go/
Because all languages and developers assume that Erlang is only about message passing. And they completely ignore literally everything else: from immutable structures to the VM providing insane guarantees (like processes not crashing the VM, and monitoring)
> But writing code like that is a very diffferent experience from, say, async JavaScript,
I write a fair amount of code in Elixir professionally and this isn't how I view it.
There are some specific Elixir/Erlang bits of ceremony you need to do to set up your supervision tree of GenServers, but then once that's done you get to write code that feels like so gle threaded "ignore the rest of the world" code. Some of the function calls you're making might be "send and message and wait for a response" from GenServers etc. but the framework takes care of that.
I wrote some driver code for an NXP tag chip. Driving the inventory process is a bit involved, you have to do a series of things, set up hardware, turn on radio, wait a bit, send data, service the SPI the whole time in parallel. With the right setup for the hardware interface I just wrote the whole thing as a sequence, it was the simplest possible code you could imagine for it. And this at the same time as running a web server, and servicing hardware interrupts that cause it to reload the state of some registers and show them to each connected web session.
> If there's a large object that more than one task needs, either you put it in a task that sends messages containing the relevant query result, or you let each task construct its own copy from the stream of messages.
When you can guarantee sole ownership, why not put that exclusive pointer in the message? I’d think that this sort of compile-time lock would be an important advantage for the type system. (I think some VMs actually do this sort of thing dynamically, but I can’t quite remember where I read about it.)
On a multiprocessor, there’s of course a balance to be determined between the overhead of shuffling the object’s data back and forth between CPUs and the overhead of serializing and shuffling the queries and responses to the object’s owning thread. But I don’t think the latter approach always wins, does it? At least I can’t tell why it obviously should.
I find the criticisms a little strange - async doesn’t imply multithreaded, and you don’t need to annotate everything shared with magic keywords if you’re async within the same thread because there’s no sharing. Only one future at a time is running on the thread and they’re within the same context.
When moving between threads I do what you suggest here and use channels to send signals rather than having a lot of shared state. Sometimes there is a crucial global state something that’s easier to just directly access, but I just write struct that manages all the Arc/RwLock or whatever other exclusive access mechanism I need for the access patterns. From the callers point of view everything is just a simple function call. When writing the struct I need to be thoughtful of sharing semantics but it’s a very small struct and I write it once and move on.
I also don’t understand their concern about making things Send+Sync. In my experience almost everything is easily Send+Sync, and things that aren’t shouldn’t or couldn’t be.
I get that sometimes you just want to wear sweatpants and write code without thought of the details, but most languages that offer that out of the box don’t really offer efficient concurrency and parallelism. And frankly you rarely actually need those things even if the “but it’s cool” itch is driving you. Most of the time a nodejs-esque single threaded async program is entirely sufficient, and a lot of the time Async isn’t even necessary or particularly useful. But when you need all these things, you probably need to hike up your sweatpants and write some actual computer code - because microseconds matter, profiled throughput is crucial, and nothing in life that’s complex is easy and anyone selling you otherwise is lying.
> Sometimes there is a crucial global state something that’s easier to just directly access, but I just write struct that manages all the Arc/RwLock or whatever other exclusive access mechanism I need for the access patterns. From the callers point of view everything is just a simple function call. When writing the struct I need to be thoughtful of sharing semantics but it’s a very small struct and I write it once and move on.
This is a recurring pattern I've started to notice with Rust: most things that repeatedly feel clunky, or noisy, or arduous, can be wrapped in an abstraction that allows your business logic to come back into focus. I've started to think this mentality is essential to any significant Rust project.
Yeah it was a bit of a block for me as well, I don’t know where it came from, but I resisted wrapping things. Reality is breaking things up into crates is encouraged anyway, and just abstracting complexity away is Not That Hard, and can usually be pretty small and concise to boot.
I think I’m used to other languages provided a lot of these abstractions or having some framework that manages it all. The frameworks in rust tend to be pretty low level (with a few notable exceptions) so perhaps that’s where it comes from.
Well for one- creating abstractions always comes with a tradeoff, so it's good to have some basic skepticism around them. But Rust embraces them, for better and worse. It equips you to write extremely safe and scalable abstractions, but it's also designed in a way that assumes you're going to use those capabilities (mainly, being really low-level and explicit by default), and so you're going to have a harder time if you avoid them
Another thing, for me, was that I came from mostly writing TypeScript, which is the opposite: the base language is breezy without abstractions, and the type system equips you to strongly-type plain data and language features, so you'll have a great time if you stick to those
But yeah, it's been interesting to see how different the answers to these questions can be in different languages!
Rust embraces abstractions because Rust abstractions are zero-cost. So you can liberally create them and use them without paying a runtime cost.
That makes abstractions far more useful and powerful, since you never need to do a cost-benefit analysis in your head, abstractions are just always a good idea in Rust.
The whole point of an abstraction is to remove complexity for the user.
So I assume you mean "implementation complexity" but that's irrelevant, because that cost only needs to be paid once, and then you put the abstraction into a crate, and then millions of people can benefit from that abstraction.
You've got a very narrow view that I'd encourage you to be more open-minded about
No abstraction is perfect. Every abstraction, when encountered by a user, requires them to ask "what does this do?", because they don't have the implementation in front of their eyes
This may be an easy question to answer- maybe it maps very obviously to a pattern or domain concept they already know, or maybe they've seen this exact abstraction before and just have to recall it
It may be slightly harder- a new but well-documented concept, or a concept that's intuitive but complex, or a concept that's simple but poorly-named
Or it may be very hard- a badly-designed abstraction, or one that's impossible to understand without understanding the entire system
But the simplest, most elegant, most intuitive abstraction in the world has nonzero cognitive cost. We abstract despite the cost, when that cost is smaller than the cost of not abstracting.
Even the costs you are talking about are a one-time cost to read the documentation and learn the abstraction. And the long-term benefits of the abstraction are far greater than the one-time costs. That's why we create abstractions, because they are a net gain. If they were not a net gain, we would simply not create them.
The whole point of abstraction is to replace the need of understanding all the details of the implementation with a more general and simpler concept. So while the abstraction itself may have a non zero cognitive cost for the end user, this cost should be lower than the cognitive cost of the full implementation that the abstraction hides. Hence the net cognitive cost of proper abstraction is negative.
Abstractions allow systems to scale. Without them, it would be impossible to work on a system that's 1M lines of code long, because you'd have to read and understand all 1M lines before doing anything.
The "zero-cost" phrase is deceptive. There's a non-zero cognitive cost to the author and all subsequent readers. A proliferation of abstractions increases the cost of every other abstraction further due to complex interactions. This is true of in all languages where the community has embraced the idea of abstraction without moderation.
Well, the intent of an abstraction is it comes at a non zero cost to the author but a substantial benefit to the user/reader. If it’s a cost to everyone why are you doing it at all?
Rust embraces zero to low cost abstraction at the machine performance level, although to get reflective or runtime adaptive abstractions you end up losing some of that zero cost as you need to start boxing and moving things into heaps and using vtables, etc. IMO this is where rust is weakest and most complex.
> There's a non-zero cognitive cost to the author and all subsequent readers.
No, the cognitive cost of a particular abstraction relative to all other abstractions under consideration can be negative.
The option of not using any abstraction doesn’t exist. If you disagree with that then I think we have to go back one step and ask what an abstraction even is.
"Zero-cost abstractions" can be a confusing term and it is often misunderstood, but it has a precise meaning. Zero-cost abstractions doesn't mean that using them has no runtime cost, just that the abstraction itself causes no additional runtime cost.
These can also be quite narrow: Rc is a zero-cost abstraction for refcounting with both strong and weak references allocated with the object on the heap. You cannot implement something the same more efficiently, but you can implement something different but similar that is both faster and lighter than Rc. You can make a CheapRc that only has strong counts, and that will be both lighter and faster by a tiny amount, or a SeparateRc that stores the counts separately on the heap, which offers cheaper conversions to/from Rc.
We're talking about the comparison between using an abstraction vs not using an abstraction.
When I said "doesn't have a runtime cost", I meant "the abstraction doesn't have a runtime cost compared to not using the abstraction".
If you want your computer to do anything useful, then you have to write code, and that code has a runtime cost.
That runtime cost is unavoidable, it is a simple necessity of the computer doing useful work, regardless of whether you use an abstraction or not.
Whenever you create or use an abstraction, you do a cost-benefit analysis in your head: "does this abstraction provide enough value to justify the EXTRA cost of the abstraction?"
But if there is no extra cost, then the abstraction is free, it is truly zero cost, because the code needed to be written no matter what, and the abstraction is the same speed as not using the abstraction. So there is no cost-benefit analysis, because the abstraction is always worth it.
The way you used it in your parent comment didn't make it clear that you were using it properly, hence my clarification. I'm honestly still not sure you've got it right, because Rust abstractions, in general, are not zero-cost. Rust has some zero-cost abstractions in the standard library and Rust has made choices, like monomorphization for generics, that make writing zero-cost abstractions easier and more common in the ecosystem. But there's nothing in the language or compiler that forces all abstractions written in Rust to be free of extra runtime costs.
I never said that ALL abstractions in Rust are zero-cost, though the vast majority of them are, and you actually have to explicitly go out of your way to use non-zero-cost abstractions.
>Rust embraces abstractions because Rust abstractions are zero-cost. So you can liberally create them and use them without paying a runtime cost.
>you never need to do a cost-benefit analysis in your head, abstractions are just always a good idea in Rust
Again though, and ignoring that, "zero-cost abstraction" can be very narrow and context specific, so you really don't need to go out of your way to find "costly" abstractions in Rust. As an example, if you have any uses of Rc that don't use weak references, then Rc is not zero-cost for those uses. This is rarely something to bother about, but rarely is not never, and it's going to be more common the more abstractions you roll yourself.
Async the keyword doesn’t, but Tokio forces all of your async functions to be multi thread safe. And at the moment, tokio is almost exclusively the only async runtime used today. 95% of async libraries only support tokio. So you’re basically forced to write multi thread safe code even if you’d benefit more from a single thread event loop.
Rust async’s set up is horrid and I wish the community would pivot away to something else like Project Loom.
You don't even need other runtimes for this. Tokio includes a single-threaded runtime and tools for dealing with tasks that aren't thread safe, like LocalSet and spawn_local, that don't require the future to be Send.
So you’re stuck choosing a single CPU or having to write send and sync everywhere. There’s a lot of use cases where you would want a thread-per-core model like Glommio to take advantage of multiple cores while still being able to write code like it’s a single thread.
> I have used Rust async extensively, and it works great. I consider Rust's Future system to be superior to JS Promises.
Sure, but it’s a major headache compared to Java VirtualThreads or goroutines
> So you’re stuck choosing a single CPU or having to write send and sync everywhere. There’s a lot of use cases where you would want a thread-per-core model like Glommio to take advantage of multiple cores while still being able to write code like it’s a single thread.
No your not, you spawn a runtime on each thread and use spawn_local on each runtime. This is how actix-web works and it uses tokio under the hood.
> So you’re stuck choosing a single CPU or having to write send and sync everywhere. There’s a lot of use cases where you would want a thread-per-core model like Glommio to take advantage of multiple cores while still being able to write code like it’s a single thread.
thread_local! exists, and you can just call spawn_local on each thread. You can even call spawn_local multiple times on the same thread if you want.
You can have some parts of your programs be multi-threaded, and then other parts of your program can be single-threaded, and the single-threaded and multi-threaded parts can communicate with an async channel...
Rust gives you an exquisite amount of control over your programs, you are not "stuck" or "locked in", you have the flexibility to structure your code however you want, and do async however you want.
You just have to uphold the basic Rust guarantees (no data races, no memory corruption, no undefined behavior, etc.)
The abstractions in Rust are designed to always uphold those guarantees, so it's very easy to do.
How is the future system superior? Is this a case of the languages type constraints being better vs non-existent? Saying something is superior doesn't really add much.
I am genuinely asking because I have little formal background in CS so "runtimes" and actual low level differences between , for instance, async and green threads mystifies me. EG What makes them actually different from the "runtime" perspective?
>but Tokio forces all of your async functions to be multi thread safe
While there are other runtimes that are always single-threaded, you can do it with tokio too. You can use a single threaded tokio runtimes and !Send tasks with LocalSet and spawn_local. There are a few rough edges, and the runtime internally uses atomics where a from-the-ground-up single threaded runtime wouldn't need them, but it works perfectly fine and I use single threaded tokio event loops in my programs because the tokio ecosystem is broader.
> But then rust also has channels. When you read about it, it talks about "messages", which to me means little objects. Like a few bytes little. This is the solution, pretty much everything I write now is just a few tasks that service some channels. They look at what's arrived and if there's something to output, they will put a message on the appropriate channel for another task to deal with. No sharing objects or anything. If there's a large object that more than one task needs, either you put it in a task that sends messages containing the relevant query result, or you let each task construct its own copy from the stream of messages.
The dream of Smalltalk and true OOP is still alive.
Why does Smalltalk constantly get credit for being true OOP? Simula was doing OOP long before Smalltalk. Most languages choose Simula style OOP, and reject the things that make Smalltalk different.
If you say Smalltalk is better OOP I might agree, but calling it "true" is not correct.
Alan Kay is generally credited with coming up with the term "object-oriented", so for better or for worse, many people defer to his definition and his embodiment of ideas when looking for a strict definition of the term.
Because the term was coined by Alan Kay, who apparently later said he probably should have called it message oriented (paraphrasing).
There's also a written conversation you can find online where he disqualifies pretty much all of the mainstream languages of being OO.
A lot of people, like you, say that OO == ADTs. Or rather, what ever Simula, C++ and Java are doing. Some will say that inheritance is an integral part of it, other's say it's all about interfaces.
But then there's people who say that Scheme and JavaScript are more object oriented than Java and C#. Or that when we're using channels or actors we're now _really_ doing OOP.
There's people who talk about patterns, SOLID, clean code and all sorts of things that you should be adhering to when structuring OO code.
Then there's people who say that OO is all about the mental model of the user and their ability to understand your program in terms of operational semantics. They should be able to understand it to a degree that they can manipulate and extend it themselves.
The referenced research also considers the publications by Kay and his team (including his theses and the Smalltalk-72 and 76 manuals) and other uses of the term. I think Kay mixes things up in retrospective; in his 1968 thesis he used the terms "object language" and "object machine", but not "object-oriented"; imagine giving your new breakthrough method a name, but then not using that name anywhere in the publication; that seems unthinkable, especially with an accomplished communicator like Kay. The first time "object-oriented" appears in a publication of his or his team is in 1978.
Right, including Smalltalk 76 and 80 onwards themselves. Remember Kay's statement "actually I made up the term object-oriented and I can tell you I did not have C++ in mind, so the important thing here is I have many of the same feelings about Smalltalk" (https://www.youtube.com/watch?v=oKg1hTOQXoY&t=636s); the reason he refers to Smalltalk this way in his 1997 talk was likely the fact that Smalltalk-80 has more in common with Simula 67 than his brain child Smalltalk-72. Ingalls explicitly refers to Simula 67 in his 2020 HOPL paper.
> and reject the things that make Smalltalk different
Which would mostly be its dynamic nature (Smalltalk-76 can be called the first dynamic OO language) and the use of runtime constructs instead dedicated syntax for conditions and loops (as it is e.g. the case in Lisp). There are a lot of dynamic OO languages still in use today, e.g. Python. Also Smalltalk-80 descendants are still in use, e.g. Pharo.
I remember picking up this sort of advice from a professor way back in college. It's a godsend. Structure the problem as data flowing between tasks and connect them up with queues, avoid sharing state. It's just a better way to deal with multithreading no matter what language you use.
There is a time any place for sharing state and data. However it is extremely complex to make that work, and so if at all possible don't. In general the only time I can't use queues is when I'm writing the queue implementation (I've done this several times - turns out there are a number of different special cases in my embedded system where it was worth it to avoid some obscure downside to the queues I already had).
When you need the absolute best performance sharing state is sometimes better - but you need a deep understanding of how your CPUs share state. A mutex or atomic write operation is almost always needed (the exceptions are really weird), and those will kill performance so you better spend a lot of time minimizing where you have them.
That was the argument for Go. But Go is not used that way. People still share and lock stuff in Go. Go is only safe for race conditions that break the memory model, not all race conditions, as Rust is.
Go cannot catch data races at compile time (like Rust) but can catch a subset of data races at run time with the race detector. Go provides imperfect, opt-in protection.
Technically, you can break Go's memory model via race conditions: Write to an interface on one thread while reading it from another and you may read the old vtable pointer but new data pointer / the other way around. Same goes for slices with data/length/capacity.
Channels with passing messages has been around as a solid way for doing async and multi threading since forever. These systems are called actor based systems. Erlang is a good example which uses it as its core. Then on the jvm there is Akka. Axon is another.
The author does mention that you should probably stop at using Threads and passing data around via channels... but then mentions the C10K problem and says that sometimes you need more... but does not answer the question that I think is begging to be asked: does using Rust async with all the complications (Arc, cloning, Mutex whatever) does actually outperform Threads/channels?? Even if it does, by how much? It would be really interesting to know the answer. I have a feeling that Threads/channels may be more performant in practice, despite the imagined overhead.
There's not a good distributed concurrent benchmark in the Techempower Web Framework benchmarks, because the Multiple Queries and Fortunes test programs don't use any parallelism or concurrency primitives to win at fast SQL queries.
https://www.techempower.com/benchmarks/#section=data-r21&tes...
Threads cannot scale at all, because you're limited to the number of threads (which is usually quite small).
Async code can scale essentially infinitely, because it can multiplex thousands of Futures onto a single thread. And you can have millions of Futures multiplexed onto a dozen threads.
This makes async ideal for situations where your program needs to handle a lot of simultaneous I/O operations... such as a web server:
Threads, at least on Linux, are much more lightweight than you seem to think. Async Rust can scale better, of course, but you're overexaggerating your case.
While the article elucidates well on the intricacies and challenges of async Rust, I feel it's crucial to note that one of Rust's core philosophies is ensuring memory safety without sacrificing performance.
The async patterns in Rust, especially with regards to data safety assurances for the compiler, are emblematic of this philosophy. Though there are complexities, the value proposition is a safer concurrency model that requires developers to think deeply about their data and execution flow. I do concur that Rust might not be the go-to for every massively concurrent userspace application, but for systems where robustness and safety are paramount, the trade-offs are justifiable. It's also worth noting that as the ecosystem evolves, we'll likely see more abstractions and libraries that ease these pain points.
Still, diving into the intricacies as this article does, gives developers a better foundational understanding, which in itself is invaluable.
Most of the rant, apart from the old man yells at function colors, is about lifetimes of arguments of async functions. And it's presenting a special case as some kind of pervasive limitation.
Async functions don't have to always own their arguments. Just the outermost future that is getting spawned on another thread has to. The rest of the async program can borrow arguments as usual. You don't need to spawn() every task — there are other primitives for running multiple futures, with borrowed data, on the same thread.
In fact, this ability for a future to borrow from itself is the reason why Rust has native await instead of using callbacks. Futures can be "self-referential" in Rust, and nothing else is allowed to.
I somewhat agree with the author, sometimes with async rust I need to figure out how to tell the compiler that yes I want to recursively call this async function. This can be a huge pain, especially because it’s not always clear what went wrong.
Other times however rust stops me from writing buggy code and where I didn’t quite understand what I was doing. In some sense it can help you understand what your software better (when the problem isn’t an implementation detail).
I get the authors frustration, I often have the same feelings. Sometimes you just want to tell rust to get out of your way.
As an aside, I think there is room for a language similar to golang with sum types and modules and be a joy.
Async is also spread through so many crates that your program will have to be async in its entirety, or at least depend on the tokio crate for a lot of things. Want a web server? Async + tokio or gtfo. Want an sql connector? You better write your own, unless you want async. Each with a different solution to the various problems async brings -- and dont even get me started on async closures and such shit, thats where hell pokes through the earth and does unholy things your compiler.
I enjoy Rust, and I love how the compiler helps me solve problems. However, the ecosystem is "async or gtfo", or "just write it yourself if you dont want async lmao", and that's not good enough.
A lot of that pain could have been avoided if the language had better primitives for async in the std or in the futures crate. Like a trait that executor must implement and a "default" blocking executor to execute async code from sync.
Right now even building a library that support multiple async runtimes is a PITA, I have done it a couple times. So you end up supporting either just tokio and maybe async-std.
In the .Net/C# world library users just expect that the library has implemented both DoThings() and DoThingsAsync(). However that is easier said that done because many of the foundational low-level IO APIs were implemented by Microsoft that first implemented IoMethod() and then implemented IoMethodAsync() when async/await became a thing in C#.
> You can break the chain by commanding the entire runtime to block on the completion of a future, but you probably shouldn’t do this pervasively since it isn’t composable. If a function blocks on a future, and that future calls a function that blocks on a future, congrats! The runtime panics!
article says you can panic if you use the pattern you show. specifically, if you call `my_normal_code()` from an async context.
is the author just talking about a quirk in tokio? or is this sort of wrapping intrinsically dangerous somehow?
> A lot of that pain could have been avoided if the language had better primitives for async in the std or in the futures crate. Like a trait that executor must implement and a "default" blocking executor to execute async code from sync.
This is one of the goals of the async working group. Hopefully, when ready, that'll make it possible to swap out async runtimes underneath arbitrary code without issues.
Async/await was a terrible idea for fixing JavaScript's lack of proper blocked threading that is currently being bolted onto every language. It splits every language and every library-ecosystem in half and will cause pains for many years to come.
Everyone who worked with multi-threading outside of JavaScript knows that using actors/communicating sequential processes is the best way to do multi-threading.
I recently found an explanation for that in Joe Armstrong's thesis. He argues that the only way to understand multi-threaded programs is writing strictly sequential code for every thread and not muddling all the code for all the threads in one place:
"The structure of the program should exactly follow the structure of the problem. Each real world concurrent activity should be mapped onto exactly one concurrent process in our programming language. If there is a 1:1 mapping of the problem onto the program we say that the program is isomorphic to the problem.
It is extremely important that the mapping is exactly 1:1. The reason for this is that it minimizes the conceptual gap between the problem and the solution. If this mapping is not 1:1 the program will quickly degenerate, and become difficult to understand. This degeneration is often observed when non-CO languages ["non concurrency-oriented", looking at you JavaScript!] are used to solve concurrent problems. Often the only way to get the program to work is to force several independent activities to be controlled by the same language thread or process. This leads to a inevitable loss of clarity, and makes the programs subject to complex and irreproducible interference errors." [0]
> Async/await was a terrible idea for fixing JavaScript's lack of proper blocked threading that is currently being bolted onto every language.
As fun as it is to hate on JavaScript, it's really interesting to go back and watch Ryan Dahl's talk introducing Node.js to the world (https://www.youtube.com/watch?v=EeYvFl7li9E). He's pretty ambivalent about it being JavaScript. His main goal was to find an abstraction around the epoll() I/O event loop that didn't make him want to tear his eyes out, and he tried a bunch of other stuff first.
> As fun as it is to hate on JavaScript, it's really interesting to go back and watch Ryan Dahl's talk introducing Node.js to the world
Agreed. JavaScript was actually my first language after TurboPascal in 1996.
I was also there listening to the first podcasts when node came out.
JavaScript is a very interesting language, especially with it's prototype memory model. And the eventloop apart from the language is interesting as well. And it's no coincidence Apple went as far as baking optimizations for JavaScript primitive operations into the M1 microcode.
But I still think multithreading is best done by using blocking operations.
NIO can be implemented on top of blocking IO as far as I know but not the other way round.
Also, sidenote, I think JavaScript's only real failure is the lack of a canonical module/import system. That error lead to countless re-implementations of buildsystems and tens of thousands of hours wasted debugging.
> Also, sidenote, I think JavaScript's only real failure is the lack of a canonical module/import system. That error lead to countless re-implementations of buildsystems and tens of thousands of hours wasted debugging.
Agreed. I don't hate on JS, in fact I think it's the best tool for the several very common use cases it targets, and I'll even defend the way objects work in it (i.e. lets me do what I want with minimal fuss). The import/require drama was annoying, though.
If you look around at the other competition at the time, it's worth noting that many other languages that already existed for decades ultimately came up with the same basic solution. In fact one of the weird things about the Node propaganda at the time was precisely that every other major scripting language tended to have not just one event-based library, but choices of event based libraries. Perl even had a metapackage abstracting several of them. It was actually a bog-standard choice, not some sort of incredible innovation.
I don't think it's a "good" solution in the abstract, but in the concrete of "I have a dynamically-typed scripting language with already over a decade of development and many more years of development that will happen before the event-based stuff is really standard", it's nearly the only choice. Python's gevent was the only other thing I saw that kinda solved the problem, and I really liked it, but I'm not sure it's a sustainable model in the end as it involves writing a package that aggressively reaches into other packages to do its magic; it is a constant game of catch-up.
I do think it's a grave error in the 2020s to adopt async as the only model for a language, though. There are better choices. And I actually exclude Rust here, because async is not mandatory and not the only model; I think in some sense the community is making the error of not realizing that your task will never have more than maybe a hundred threads in it and a 2023 computer will chomp on that without you noticing. Don't scale for millions of concurrent tasks when you're only looking at a couple dozen max, no matter what language or environment you're in. Very common problem for programmers this decade. It may well be the most impactful premature optimization in programming I see today.
> Don't scale for millions of concurrent tasks when you're only looking at a couple dozen max, no matter what language or environment you're in. Very common problem for programmers this decade.
And also with fibers/virtual threads (project loom) you can actually have a million threads using blocking hand-off on one machine. So the performance argument is kind of gone.
It’s so interesting that the async/await stuff basically makes his presentation points meaningless? if you’re just using the async/await why use the callback style in the first place…
but I get it, you can always go back to the promises and callbacks if you want.
He was strongly advocating callbacks with node.js, which is understandable given the time. But a few years later when he wrote some Go code, he said that's a better model for networked servers (sorry no reference right now, it was in a video I watched, not sure which one)
JS callbacks are indeed better than C callbacks because you can hold onto some state. Although I guess the capture is implicit rather than explicit, so some people might say it's more confusing.
I'm pretty sure Joyent adopted and funded node.js because they were doing lots of async code in C, and they liked the callback style in JavaScript better. It does match the kind of problems that Go is now being used for, and this was pre-Go.
But anyway it is interesting how nobody really talks about callbacks anymore. Seems like async/await has taken over in most languages, although I sorta agree with the parent that it could have been better if designed from scratch.
async/await actually originated in C#, not Javascript. C#'s author, Anders Hejlsberg also authored Typescript. Typescript's additional features like classes, arrow functions and async/await eventually crept into ES6+.
I actually think it was a great solution in JS/TS given it's a single threaded event loop. The lower level the language the worse of an abstraction it is though. So I think most of the complaints here about async Rust are valid.
Promise/Future style of async is just a bad idea regardless of language.
It was used because of ineptitude of languages where it become popular, and its far easier to implement into GC-less languages than message-passing-based asynchronous, but it's just misery to write code in. I'd prefer to suffer Go ineptitudes just to use bastardised message passing called channels there rather than any of the Python/JS/Rust async.
Yes. That atomized model of concurrency where your state goes everywhere and you somehow collect it back at some point was always (literally) the textbook example of how not to do it.
It was created to be an improvement over the Javascript situation, and somehow every language that had a sane structure adopted it as if it was not only good, but the way to do things. This is insane.
And yet, people are going to use async in Rust. The feature has already proven itself useful long ago in other languages, beyond the timespan a fad could survive. Everyone started out doing it the other way and got sick of it.
On the voodoo ridden land that is software development, we have plenty of clearly harmful fads that are much older than Rust and yet practiced everywhere.
Up to now, rust async has lasted for less time than the NoSQL craziness. I'm hard pressed to think of any large fad that lasted less than it.
Async is much older in other languages. It's new in Rust, and time will tell, but I don't see it playing out differently this time.
Btw, the turnaround time is longer with a database, which often forms the foundation of a system. NoSQL bandwagoning was so destructive in part because of how long it looked like a good idea each time. Same with ORMs.
The C# implementation is clearly an attempt of putting type-safety over exactly the same implementation JS promises use. Done because MS wanted to port the same behavior.
- the C# implementation predates even Promises in JS, so it is not "the same implementation" and your implication that C# was inspired by JS as opposed to the other way around is false. More background: [0]
- Typescript works fine with the JS implementation so any differences aren't for type safety reasons, but largely because C# has a multithreaded event loop unlike JS
Also promises (or "futures" as they're called elsewhere) aren't unique to any language. They're used in lots of places that predate both C# and JS's use, for example the twisted framework in Python.
You're implying people went from writing C# to JS code willingly and not just wrote it but went on "improving" the ecosystem and I just don't believe there are people insane enough to do it willingly so "it was invented separately in both instances" is far more likely.
I can't tell if you're talking about the language designers or users.
For language designers I point out in my other comment that Anders authored both C# and TS. TS' influence on ES6 is documented publicly.. heck TC39 has an open proposal to add type annotations to JS now!
As for users, any dev that touches frontend has to write JS, unless you're purely a mobile or desktop shop (even then, there's electron). So yes, I think tons of folks willingly write C# and JS/TS. I'm certainly one (though write more Python than both these days). Was I an early adopter of async in Python because of my familiarity of it in C#/JS? You bet I was. Maybe I'm "insane."
> concurrency where your state goes everywhere and you somehow collect it back at some point was always (literally) the textbook example of how not to do it.
can you tell why it is not how not to do it in your opinion? What are the obvious issues with this approach?
> Promise/Future style of async is just a bad idea
JVM's futures are a joy to work with compared to JS's promises (or Kotlin's coroutines for that matter). While similar, I don't think you can conflate them.
You're conflating the idea of using channels with green threads. They are different, you can easily use channels with async/await and global state/mutexes with green threads.
I have been using "async/await is bad, use {feature name[0]}" as a litmus test for people who are generally bad at programming, especially so at concurrent flavour of such.
Sure, Rust is certainly verbose and very strict how the ownership rules apply in the context of async, but this is a hard constraint of its memory safety model.
We could probably do better while retaining all performance but this is by far one of the best implementations. Another example of nice to use async/await is C# which trades performance/memory (state has to be boxed if it is to live across continuations) for convenience (you just write it naturally without worrying about underlying behavior).
There is a reason Rust toyed with "green threads" at its inception but decided against such. The only popular languages of today that do these are Go and Java (which basically forced to do this because you can't go async without introducing the feature early in the lifecycle of the language, and the authors of project Loom are simply wrong with their excuses why this is superior to async/await).
Async/await is here to stay and is the right abstraction, git good, and it's not even difficult to use anyway.
[0] where feature name is green threads, not doing concurrency at all, doing it manually, etc.
If someone is foolish enough to implement a websocket library that insists on performing I/O by calling the methods on the language's generic reader and writer interfaces (rather than just operating on buffers the user passes in or whatever), why should they need to implement it a second time to add async? There are plenty of languages in which they do not need to do this, like Python or Lua.
How does that relate to async/await? This is an arbitrary implementation choice. Generic reader/writer in Rust wouldn't matter since it will get optimized away, only leaving cooperative multitasking code if the buffer were to be filled by another thread/producer/source.
If you write a library that implements websockets on top of the Read trait or on top of sockets, is it *also* an async websocket library, with no additional code and without containing any instances of the async keyword? In many languages the answer is yes.
I guess I should specify that this is true even in a single-threaded context and even without any additional buffers or whatever.
Disliking async/await does not make someone "generally bad at programming". This is a childish ad-hominem mindset that has no place in technical debates. Rust's decision to adopt async/await over green threads was intended to keep the runtime lean, not because it is an inherently better abstraction. Java certainly could have async/await syntactic sugar around its existing futures api, but project loom has greater ambitions by retrofitting asynchronous IO onto the existing thread api. The authors are certainly not "simply wrong" for this decision.
Sure, it is "childish and ad-hominem", it is also efficient and usually works. Also, non-async/await solutions for concurrency and asynchrony result in a more verbose, complex and bloated syntax, having to invent fancy terms like "structured concurrency" for use cases which in C# are simply expressed by awaiting the task at the point of its consumption and not declaration (no special methods required) or in Rust via simple, albeit for a different scenario, join!().
var user = try alloc.create(@Frame(service.GetUser));
user.* = async service.GetUser(id);
defer alloc.destroy(user);
var promos = try alloc.create(@Frame(service.GetPromotions));
promos.* = async service.GetPromotions(category);
defer alloc.destroy(promos);
var eligible = GetEligibility(await user.*, await promos.*);
but the claimed difference is not present in this code. It is that GetUser and GetPromotions do not themselves know whether they are async. The author of service is able to just take some readers and writers or some types or objects implementing some protocols and use them, then service's methods inherit the async-ness of its dependencies.
async/await and structured concurrency are two different things, Swift has an async/await syntax but is still structured concurrency.
The point of async/await is that it converts your function into a reentrant state machine (in a different way than compiling a sync function already turned it into a state machine.) The problem with the usual design is that it uses futures, which are bad because they have dynamic lifetimes.
> Async/await is here to stay and is the right abstraction, git good, and it's not even difficult to use anyway.
It's probably the right abstraction for Haskell, or any other language that works well with functional programming, lambdas and monads. Loom is a better fit for Java. Rust also would have probably been better off with something else. Effect handlers might have been a good choice.
> At this scale, threads won’t cut it—while they’re pretty cheap, fire up a thread per connection and your computer will grind to a halt.
Maybe in the 2000's but I feel this reasoning is no longer valid in 2023 and should be put to rest.
10k problem.. Wouldn't modern computing not work if my Linux box couldn't spin up 10k threads? Htop says I'm currently at 4,000 threads on an 8 core machine.
> 10k problem.. Wouldn't modern computing not work if my Linux box couldn't spin up 10k threads? Htop says I'm currently at 4,000 threads on an 8 core machine.
By the 2010s the problem had been updated to C10M. The people discussing it (well, perhaps some) aren't idiots and understand that the threshold changes as hardware changes.
Also, the issue isn't creating 10k threads it's dealing with 10k concurrent users (or, again, a much higher number today).
The context switch for threads remains very expensive. You have 4,000 threads but that's lots of different processes spinning up their own threads. it's still more efficient to have one thread per core for a single computational problem, or at most one per CPU thread (often 2 threads per core now). You can test this by using something like rayon or GNU parallel using more threads than you have cores. It won't go faster, and after a certain point, it goes slower.
The async case is suited to situations where you're blocking for things like network requests. In that case the thread will be doing nothing, so we want to hand off the work to another task of some kind that is active. Green threads mean you can do that without a context switch.
Since that time, context switching changed from a O(log(n)) operation to an O(1) one.
I have no doubt that having a thread per core and managing the data with only non-blocking operations is much faster. But I'm pretty current machines can manage a thousand or so threads locked almost the entire time just fine.
> Since that time, context switching changed from a O(log(n)) operation to an O(1) one.
I'm not sure how that's relevant here, if for example something takes 1ms and I do it 1000 times a second, I'm using 1000 ms of CPU time vs not doing it at all.
So if you want to use big o notation in this context it should be O(n) where n is the number of context switches, because you are not comparing algorithms used to switch between threads but you are comparing doing context switch or not doing it at all.
> The context switch for threads remains very expensive
It got even more expensive in recent years after all the speculative execution vulnerabilities in CPUs, so now you have additional logic on every context switch with mitigations on in kernel.
> The numbers reported here paint an interesting picture on the state of Linux multi-threaded performance in 2018. I would say that the limits still exist - running a million threads is probably not going to make sense; however, the limits have definitely shifted since the past, and a lot of folklore from the early 2000s doesn't apply today. On a beefy multi-core machine with lots of RAM we can easily run 10,000 threads in a single process today, in production. As I've mentioned above, it's highly recommended to watch Google's talk on fibers; through careful tuning of the kernel (and setting smaller default stacks) Google is able to run an order of magnitude more threads in parallel.
so, in that benchmark, context switch is comparable to copying 64k mem, which is kinda significant, I run some heavy load database with few hundreds threads, and see that it does 100k context switching per sec some times.
> Maybe in the 2000's but I feel this reasoning is no longer valid in 2023 and should be put to rest.
So do we discard existing ways of making software more efficient because we can be more wasteful on more recent hardware? What if we could develop our software such that 2000s computers are still useful, rather than letting those computers become e-waste?
This is a pretty interesting article... and I generally agree with the pain points... but I don't really like the conclusion
* While the author states that not many apps "need" high concurrency in userspace... I would invert that and say that we may be missing so much performance, new potential applications, etc because highly concurrent code is so hard to get right. One bit of evidence of this (to me at least) is how often in my career I have had to scale things up due to memory or other resource limitations and not CPU. And when it is CPU, so often looking into it more finds bugs with concurrency that are the root cause or at least exacerbate the issue
* While I completely agree that rust is not easy with async and have myself poked around at which magical type things I need to do each time I have touched async rust code, I don't really like the suggestion being to "go use a different language", first, because if you are picking up rust, you (IMHO) should have a very good reason to already have chosen it. Rust is not easy enough or ubiquitous enough that you should be choosing it "just for fun" and your reason for using Rust should be compelling enough that you (right now) are willing to put in the effort to learn async when you need it
* What the other mentions in the body of the article, but I think is more of what my suggestion would be: don't use async unless you need it!. While I would love to see Rust (and think it should) evolve to the point where async is "easy", maybe we instead just need to get more pragmatic in what is taught and written about. I think when people start Rust they want to use all the fanciness, which includes async, and while some of that is just programmers, I think it is also how tutorials, docs, and general communication about a programming language happens where we show the breadth of capability, rather than the more realistic learning path, which leads people to feel like if they don't use async, they aren't doing it right
Finally, I do really hope Rust keeps working on the promise of these zero cost abstractions that can really simplify things... but if that doesn't work, I am at least hopeful of what people can build on top of the rust featureset/toolchain to help make things like async more realistic to be the default without the need for a complex VM/runtime.
Anyone with electrical engineering know if pata vs. sata cables is a good analogy for async vs. sync?
I know parallel ATA cables were all the rage. They had a higher theoretical throughput when compared with serial ATA cables but there was too much cross-talk involved to make it actually faster in the end so now we have serial ATA cables everywhere with much higher throughput than parallel ATA cables could ever achieve.
Should we move back away from parallelism and focus on handling synchronous stuff faster instead?
We're dealing with fundamental limitations of I/O though, which is causing the delays.
I want to write stuff to disk (SSD these days). I can issue a request, then have to wait tens to hundreds of milliseconds (in the average case, the worst case can be far longer) for that request to finish and let me know that my I/O request succeeded or failed. There's no getting around that with present-day technology.
The situation is worse and even less reliable with network I/O. If you are talking to a server in another continent, the speed of light determines the minimum of time I hear back from it, even if it (and all the intermediary network links) are lightly loaded and functioning perfectly.
PATA vs. SATA is a somewhat limited metaphor; PATA had a number of limitations such as the inability to hot swap hardware as well as using wide ribbon cables that made it largely obsolete. In contrast, both sync and async programming have reasonable applications; we're likely using both for the foreseeable future. The best EE analogy I can think of is using hyperthreading to execute multiple processes on a single core vs scheduling each thread to a separate core, but that's less a metaphor and more of a simplified model of what async vs sync is actually doing.
> Should we move back away from parallelism and focus on handling synchronous stuff faster instead?
Rust already has excellent handling of synchronous computation, given that it can meet/sometimes exceed equivalent performance in C. The problem is when you're I/O or network bound; you can either throw threads at the problem (and by extension throw memory at the problem for the thread stacks) or use async programming.
Async Rust is especially problematic in the enterprise world where large software is built out of micro-services connected through RPC.
Typically, if you want to build something with Rust, it'll have to use async, at least because gRPC and the like are implemented that way. So the vanilla (and excellent, IMO) Rust language doesn't exist there. Everything is async from the get-go.
> Async Rust is especially problematic in the enterprise world where large software is built out of micro-services connected through RPC.
A weird way to use Rust since you can do a lot of messaging within the process, and use the computing power much more efficiently.
RPC is essentially messaging and message-passing. Message-passing is a way to avoid mutable shared state - this is the model with which Go became successful.
RPC surely has its use but message passing is another, and very often inferior, solution to the problem set where Rust has excellent own solutions for.
> We want to use the whole computer. Code runs on CPUs, and in 2023, even my phone has eight of the damn things. If I want to use more than 12% of the machine, I need several cores.
Isn't that already, in this strong generality, an almost always wrong assumption?
Sure, one can do massively parallel or embarrassingly parallel computation.
Sure, graphic cards are parallel computers.
Sure, OS kernels use multiple cores.
Sure, languages and concepts like Clojure exist and work - for a specific domain, like web services (and for that, Clojure works fascinatingly well).
But there are many, even conceptually simple algorithms which are not easy to parallelize. There is no efficient parallel Fast Fourier Transform I know of.
And there are even different degrees of parallelization. Some things will scale almost linearly to CPU cores, some will share a little state and see diminishing returns, some will share a lot of state and maybe only make good use of 2 cores, and it'll all depend on the hardware too.
Anyone know why there isn't a single type/interface that allows for consumers to supply any of Arc, Rc etc boxed values?
I haven't investigated it deeply, but I was developing something in Rust, and whether something needs to be threadsafe or not is entirely on the consumer's use case... bad separation of concerns for the provider of a generic interface to have to specify the specific type of boxed value. 100% fine if the behavior in this case is to pre-allocate the max possible boxed type memory requirement.
This is the only thing I was really frustrated with in Rust
Borrowing the value didn't work for this case, but forget the reasoning at the moment. This is a struct that may or may not be used in threaded code. Code is highly polymorphic and involves a few traits
Using Arc everywhere solves it, but dumb and inefficient for non threaded use cases. Maybe compiler optimizes this though, who knows. Semantically it's wrong though.
Honestly forget the specifics enough at this point to discuss so I'll drop it haha.
Was just curious whether somebody else was tracking this, or there was a known workaround. I think it's something the language will eventually support. I saw other threads on rustlang asking for the same thing, and best I saw was some sort of enum style hack representing the boxed types to emulate it
> You might say this isn’t a fair comparison—after all, those languages hide the difference between blocking and non-blocking code behind fat runtimes, and lifetimes are handwaved with garbage collection. But that’s exactly the point! These are pure wins when we’re doing this sort of programming.
I have a lot of respect for Discord's technical decisions. They know when to do things the bland way and when to use more specialized technologies. Note that that article also praises async Rust.
I got no horse in this race -- I like both Golang and Rust and use them for different things -- but as far as I can tell, Golang's GC has improved a lot.
Not sure where does it stand on a global competition rating board (if there's even such a thing) but it's pretty good. I've never seen it crap the bed, though I also never worked at the scale of Twitch and Discord.
Yes, async is effectively a much harder version of Rust, and it's regrettable how it's been shoved down the throats of everyone, while only 1% of projects using it really need it. Hover, async is also amazing in these 1% of cases when it's useful.
If you have a service that handles massive amounts of network calls at the core (think linkerd, nginx, etc.), or you want to have a massive amount of lightweight tasks in your game, or working on an embedded software where you want cooperative concurrency, async Rust is an amazing super-power.
Most system/application level things is not going to need async IO. Your REST app is going to be perfectly fine with a threadpool. Even when you do need async, you probably want to use it in a relatively small part of your software (network), while doing most of the things in threads, using channels to pass work around between async/blocking IO parts (aka hybrid model).
Rust community just mindlessly over-did using async literally everywhere, to the point where the blocking IO Rust (the actually better UX one) became a second class citizen in the ecosystem.
Especially visible with web frameworks where there is N well designed async web frameworks (Axum, Wrap, etc.) and if you want a blocking one you get:
tiny_http, absolute bare bones but very well done
rouille - more wholesome, on top of tiny_http, but APIs feel very meh comparing to e.g. Axum
astra - very interesting but immature, and rather barebones
The argument here is that Rust chose to implement coroutines the wrong way. It went the route of stackless coroutines that need async/await and colored functions. This creates all the friction the article laments over.
But it also praises Go for its implementation, which is also based on a coroutine of a different kind. Stackful coroutines, which do not have any of these problems.
Rust considered using those (and, at first, that was the project's direction). Ultimately, they went to the stackless operation model because stackfull coroutine requires a runtime that preempts coroutines (to do essentially what the kernel does with threads). This was deemed too expensive.
Most people forget, however, that almost no one is using runtime-free async Rust. Most people use Tokio, which is a runtime that does essentially everything the runtime they were trying to avoid building would have done.
So we are left in a situation where most people using async Rust have the worst of both worlds.
That being said, you can use async Rust without an async runtime (or rather, an extremely rudimentary one with extremely low overhead). People in the embedded world do. But they are few, and even they often are unconvinced by async Rust for their own reasons.
Threads are driven by the OS. Something needs to drive couritines, so there's no way around needing some (even rudimentary, like in embedded) executor. But to be a versatile and universal systems language, Rust can't just build-in executor into a language.
I think that stackless coroutines are better than stackfull, in particular for Rust. Everything was done correctly by the Rust team.
Again, this is all fair and good, as long as people understand the tradeoff and make good technical decisions around. If they all jump on async bandwagon blind o the obvious limitations, we get where Rust ecosystem is now.
Well people who jumped on async bandwagon are deeply involved in Rust community. So if they do something, others have to assume they are doing it right.
> Rust considered using those (and, at first, that was the project's direction). Ultimately, they went to the stackless operation model because stackfull coroutine requires a runtime that preempts coroutines (to do essentially what the kernel does with threads). This was deemed too expensive.
Stackful coroutines don't require a preemptive runtime. I certainly hope that we didn't end up with colored functions in Rust because of such a misconception.
They often implement soft preemption. Tokio and others like Glommio do. Usually, it's based on interrupts. The runtime schedules a timer to fire an interrupt, and some code is injected into the interrupt handler.
This is used to keep track of task runtime quotas so they can yield as soon as possible afterward.
This is the same technique used in Go and many others for preemption. If you don't add this, futures that don't yield can run forever, stalling the system.
You are right that it is not strictly necessary, but in practice, it is so helpful as a guard against the yielding problem that it's ubiquitous.
> I certainly hope that we didn't end up with colored functions in Rust because of such a misconception.
There's nothing buggy about a future that never yields because it can always make progress, but people prefer that a runtime doesn't let all other execution get starved by one operation. That makes it a problem that runtimes and schedulers work to solve, but not a bug that needs to be prevented at a language level. A runtime that doesn't solve it isn't buggy, but probably isn't friendly to use, like how Go used to have problems with tight loops and they put in changes to make them cause less starvation.
As someone else said, it is not, strictly speaking, a bug. If your server receives a request that requires very computationally expensive work, is it okay to delay every other request on that core? That's probably not okay, and it'll show in your latency distribution.
Folks would rather have every future time sliced so that other tasks get some CPU time in a ~fair way (after all, there is no concept of task priority in most runtime).
But you're right: it isn't required, and you could sprinkle every loop of your code with yielding statements. But knowing when to yield is impossible for a future. If nothing else is running, it shouldn't yield. If many things are running but the problem space of the future is small, it probably shouldn't yield either, etc.
You simply do not have the necessary information in your future to make an informed decision. You need some global entity to keep track of everything and either yield for you or tell you when you should yield. Tokio does the former, Glommio does the latter.
It gets even more complex when you add IO into the mix because you need to submit IO requests in a way that saturates the network/nvme drives/whatever. So if a future submits an IO request, it's probably advantageous to yield immediately afterward so that other futures may do so as well. That's how you maximize throughput. But as I said, that's a very hard problem to solve.
Trying to solve the problem by frequently invoking signal handlers will also show in your latency distribution!
I guess if someone wants to use futures as if they were goroutines then it's not a bug, but this sort of presupposes that an opinionated runtime is already shooting signals at itself. Fundamentally the language gives you a primitive for switching execution between one context and another, and the premise of the program is probably that execution will switch back pretty quickly from work related to any single task.
I read the blog about this situation at https://tokio.rs/blog/2020-04-preemption which is equally baffling. The described problem cannot even happen in the "runtime" I'm currently using because io_uring won't just completely stop responding to other kinds of sqe's and only give you responses to a multishot accept when a lot of connections are coming in. I strongly suspect equivalent results are achievable with epoll.
>Trying to solve the problem by frequently invoking signal handlers will also show in your latency distribution!
So just like any other kind of scheduling? "Frequently" is also very subjective, and there are tradeoffs between throughput, latency, and especially tail latency. You can improve throughput and minimum latency by never preempting tasks, but it's bad for average, median, and tail latency when longer tasks starve others, otherwise SCHED_FIFO would be the default for Linux.
You've misunderstood the problem somehow. There is definitely nothing about tokio (which uses epoll on Linux and can use io_uring) not responding in there. io_uring and epoll have nothing to do with it and can't avoid the problem: the problem is with code that can make progress and doesn't need to poll for anything. The problem isn't unique to Rust either, and it's going to exist in any cooperative multitasking system: if you rely on tasks to yield by themselves, some won't.
Yes. Industries that care about latency take some pains to avoid this as well, of course.
> io_uring and epoll have nothing to do with it and can't avoid the problem: the problem is with code that can make progress and doesn't need to poll for anything.
They totally can though? If I write the exact same code that is called out as problematic in the post, my non-preemptive runtime will run a variety of tasks while non-preemptive tokio is claimed to run only one. This is because my `accept` method would either submit an "accept sqe" to io_uring and swap to the runtime or do nothing and swap to the runtime (in the case of a multishot accept). Then the runtime would continue processing all cqes in order received, not *only* the `accept` cqes. The tokio `accept` method and event loop could also avoid starving other tasks if the `accept` method was guaranteed to poll at least some portion of the time and all ready handlers from one poll were guaranteed to be called before polling again.
This sort of design solves the problem for any case of "My task that is performing I/O through my runtime is starving my other tasks." The remaining tasks that can starve other tasks are those that perform I/O by bypassing the runtime and those that spend a long time performing computations with no I/O. The former thing sounds like self-sabotage by the user, but unfortunately the latter thing probably requires the user to spend some effort on designing their program.
> The problem isn't unique to Rust either, and it's going to exist in any cooperative multitasking system: if you rely on tasks to yield by themselves, some won't.
If we leave the obvious defects in our software, we will continue running software with obvious defects in it, yes.
>This sort of design solves the problem for any case of "My task that is performing I/O through my runtime is starving my other tasks."
Yeah, there's your misunderstanding, you've got it backwards. The problem being described occurs when I/O isn't happening because it isn't needed, there isn't a problem when I/O does need to happen.
Think of buffered reading of a file, maybe a small one that fully fits into the buffer, and reading it one byte at a time. Reading the first byte will block and go through epoll/io_uring/kqueue to fill the buffer and other tasks can run, but subsequent calls won't and they can return immediately without ever needing to touch the poller. Or maybe it's waiting on a channel in a loop, but the producer of that channel pushed more content onto it before the consumer was done so no blocking is needed.
You can solve this by never writing tasks that can take "a lot" of time, or "continue", whatever that means, but that's pretty inefficient in its own right. If my theoretical file reading task is explicitly yielding to the runtime on every byte by calling yield(), it is going to be very slow. You're not going to go through io_uring for every single byte of a file individually when running "while next_byte = async_read_next_byte(file) {}" code in any language if you have heap memory available to buffer it.
Reading from a socket, as in the linked post, is an example of not performing I/O? I'm not familiar with tokio so I did not know that it maintained buffers in userspace and filled them before the user called read(), but this is unimportant, it could still have read() yield and return the contents of the buffer.
I assumed that users would issue reads of like megabytes at a time and usually receive less. Does the example of reading from a socket in the blog post presuppose a gigabyte-sized buffer? It sounds like a bigger problem with the program is the per-connection memory overhead in that case.
The proposal is obviously not to yield 1 million times before returning a 1 meg buffer or to call read(2) passing a buffer length of 1, is this trolling? The proposal is also not some imaginary pie-in-the-sky idea; it's currently trading millions of dollars of derivatives daily on a single thread.
You're confusing IO not happening because it's not needed with IO never happening. Just because a method can perform IO doesn't mean it actually does every time you call it. If I call async_read(N) for the next N bytes, that isn't necessarily going to touch the IO driver. If your task can make progress without polling, it doesn't need to poll.
>I'm not familiar with tokio so I did not know that it maintained buffers in userspace
Most async runtimes are going to do buffering on some level, for efficiency if nothing else. It's not strictly required but you've had an unusual experience if you've never seen buffering.
>filled them before the user called read()
Where did you get this idea? Since you seem to be quick to accuse others of it, this does seem like trolling. At the very least it's completely out of nowhere.
>it could still have read() yield and return the contents of the buffer.
If I call a read_one_byte, read_line, or read(N) method and it returns past the end of the requested content that would be a problem.
>I assumed that users would issue reads of like megabytes at a time and usually receive less.
Reading from a channel is the other easy example, if files were hard to follow. The channel read might implemented as a quick atomic check to see if something is available and consume it, only yielding to the runtime if it needs to wait. If a producer on the other end is producing things faster than the consumer can consume them, the consuming task will never yield. You can implement a channel read method that always yields, but again, that'd be slow.
>The proposal is obviously not to yield 1 million times before returning a 1 meg buffer, is this trolling
No, giving a illustrative example is not trolling, even if I kept the numbers simple to make it easy to follow. But your flailing about with the idea of requiring gigabyte sized buffers probably is.
> You're confusing IO not happening because it's not needed with IO never happening. Just because a method can perform IO doesn't mean it actually does every time you call it. If I call async_read(N) for the next N bytes, that isn't necessarily going to touch the IO driver.
Maybe you can read the linked post again? The problem in the example in the post is that data keeps coming from the network. If you were to strace the program, you would see it calling read(2) repeatedly. The runtime chooses to starve all other tasks as long as these reads return more than 0 bytes. This is obviously not the only option available.
I apologize for charitably assuming that you were correct in the rest of my reply and attempting to fill in the necessary circumstances which would have made you correct
Actually, no, I misread it trying to make sense of what you were posting so this post is edited.
This is just mundane non-blocking sockets. If the socket never needs to block, it won't yield. Why go through epoll/uring unless it returns EWOULDBLOCK?
For io_uring all the reads go through io_uring and generally don't send back a result until some data is ready. So you'll receive a single stream of syscall results in which the results for all fds are interleaved, and you won't even be able to write code that has one task doing I/O starving other tasks. For epoll, polling the epoll instance is how you get notified of the readiness for all the other fds too. But the important thing isn't to poll the socket that you know is ready, it's to yield to runtime at all, so that other tasks can be resumed. Amusingly upon reading the rest of the blog post I discovered that this is exactly what tokio does. It just always yields after a certain number of operations that could yield. It doesn't implement preemption.
Honestly I assumed you had read the article and were just confused about how tokio was pretending to have preemption. Now you reveal you hadn't read the article so now I'm confused about you in general, it seems like a waste of time. But I'm glad you're at least on the same page now, about how checking if something is ready and yielding to the runtime are separate things.
> This is the same technique used in Go and many others for preemption. If you don't add this, futures that don't yield can run forever, stalling the system.
Tokio and glommio using interrupts is ironically another misconception. They're cooperatively scheduled so yes, a misbehaving blocking task can stall the scheduler. They can't really interrupt an arbitrary stackless coroutine like a Future due to having nowhere to store the OS thread context in a way that can be resumed (Each thread has its own stack, but now it's stackful with all the concerns of sizing and growing. Or you copy the stack to the task but now have somehow to fixup stack pointers in places the runtime is unaware).
For better or worse, when faced with choices like this Rust has consistently decided to make sure it's workable for the lowest-level usecases (embedded, drivers, etc). I respect the consistency, and I appreciate that it's focused on an under-served market, especially compared to eg. web applications (an over-served market, if anything), even if it's sometimes a bummer for me personally
> because stackfull coroutine requires a runtime that preempts coroutines
I've used stackful coroutines many times in many codebases. It never required or used a runtime or preemption. I'm not sure why having a runtime that preempts them would even be useful, since it defeats the reason most people use stackful coroutines in the first place.
"stackful coroutines" the control-flow primitive is cumbersome to build on top of "green threads" but for use cases that are mostly about blocking on lots of distinct I/O calls at the same time people may be indifferent between these two things. These conversations are often muddled because the feature shipped most often is called "async" and not called "jump to another stack please" :(
Lua comes with this sort of thing. OCaml, Python, and C have libraries providing this sort of thing in decreasing order of adoption.
Python also comes with 2 features that seem to be stackless coroutines with attached syntax ceremonies, but one of those 2 features is commonly used with a hefty runtime instead of being used for control flow. JavaScript comes with 2 features named similarly to those of Python, but only one of them seems to be "runtime-free" stackless coroutines.
It was always C++ for some type of high-performance data processing engine. Around half the stackful coroutine implementations were off-the-shelf libraries (e.g. Boost::Context) and the other half were purpose-built from scratch, depending on the feature requirements. The typical model is that you have stackful coroutines at a coarse level, e.g. per database query, which may dispatch hundreds of concurrent state machines. All execution and I/O scheduling is explicitly done by the software, which enables some significant runtime optimizations.
If coroutines can be preempted then it introduces a requirement for concurrency control that otherwise doesn't need to exist and interferes with dynamic cache locality optimizations. These are some of the primary benefits of using stackful coroutines in this context.
Being able to interrupt a stackful coroutine has utility for dealing with an extremely slow or stuck thread but you want this to be zero-overhead unless the thread is actually stuck. In most system designs, the time required to traverse any pair of sequential yield points is well-bounded so things getting "stuck" is usually a bug.
Letting end-users inject arbitrary code into these paths at runtime does require the ability to interrupt the thread but even that is often handled explicitly by more nuanced means than random preemption. Sometimes "extremely slow" is correct and expected behavior, so you have to schedule around it.
Rust chose to drop the green thread library so that it could have no runtime, supporting valuable use cases for Rust like embedding a Rust library into a C binary, which we cared about. Go is not really usable for this (technically it's possible, but it's ridiculous for exactly this reason). So those sorts of users are getting a lot of benefit from Rust not having a green threading runtime. As are any users who are not using async for whatever reason.
However, async Rust is not using stackless coroutines for this reason - it's using stackless coroutines because they achieve a better performance profile than stackful coroutines. You can read all about it on Aaron Turon's blog from 2016, when the futures library was first released:
It is not the case that people using async Rust are getting the "worst of both worlds." They are getting better performance by default and far greater control over their runtime than they would be using a stackful coroutine feature like Go provides. The trade off is that it's a lot more complicated and has a bunch of additional moving parts they have to learn about and understand. There's no free lunch.
People love(d) rust because it’s a pleasant language to write code for while also being insanely performant. Async is taking away the first point and making it miserable to write code for. If this trend continues, it’ll ultimately destroy the credibility of the language and people will choose other languages. The proposers of async did not take this into account when they were proposing async
Naive question, since I tried my hand at rust years ago, but haven't looked at it since: isn't it possible to write another crate to build go-like channels? A kind of "write, then lose the reference" function call that places a value on a queue, and an accompanying receiver. That could make life easier for "normal" software development.
There are many such primitives in Rust (including one in the standard library). And it's effectively the default, the only annoying thing is the libraries which use async (it is possible to just wrap the async code in sync code, just a little annoying. But I think it's what most users of the language should do.)
But "most" users can live with a bit of overhead in return for safe parallelism. It's just a handful that wants to squeeze the last bit of power out of a CPU.
The other day, Intel revealed a processor with 66 thread support per core. 64 of those threads were called "slow", because there's no prefetching and speculative execution, as they are supposed to be waiting (mainly for memory, but networking could be another option). Perhaps very many cheap hardware threads is a way out of this.
That is my point. For most people just using synchronous code with threads is the best option. Async only shines if you really want to push your I/O relative to your compute (which is becoming more of a challenge on modern hardware as I/O bandwidth is rapidly expanding compared to compute), or if you need to keep track of an extremely large number of tasks with low memory overhead. Starting off with "I'm writing a network application, I should use async" is likely already a mistake, especially in Rust.
Can you admit that you failed in making it a pleasant experience to write async, especially for library authors? I don’t think it’s too late to admit failure and implement something like May https://github.com/Xudong-Huang/may
Async await is polluting rust. What was once my favorite language is now a pain in the ass. And I am not the only person who feels this way. There’s no shame in pivoting
Is there any reason to use async when your platform supports virtual threads?
I ask as someone who uses java and is about to rewrite a bunch of code to be able to chuck the entire async paradigm into the trash can and use a blocking model but on virtual threads where blocking is ok.
Virtual threads or green threads, etc., are all names for the same thing: stackful coroutines. I would say yes! If your language/platform/runtime supports them, that should definitely be your starting point.
Not OP, but synchronous code is much, much easier to understand and write than asynchronous code. What Java is doing is making synchronous code have all the advantages of asynchronous code by making blocking a Thread become a cheap operation (instead of blocking a real OS Thread), making the whole benfit of async code go away while getting rid of async's difficulties, specially in a language that doesn't have async/await (which makes async code "look" synchronous - but in Rust, as this blog post shows, that is not really the case).
> Yes, async is effectively a much harder version of Rust, and it's regrettable how it's been shoved down the throats of everyone, while only 1% of projects using it really need it.
Yes. I just noticed that Tokio was pulled into my program as a dependency. Again. It's not being used, but I'm using a crate which has a function I'm not using which imports reqwest, which imports h2, which imports tokio.
Concurrency's correct primitive is Hoare's Communicating Sequential Processes mapped onto green threads. Some languages that have it right are Java (since JDK17 - Java Virtual Threads), Go, Kotlin.
Virtual threading is fun and all until you find out SimpleDateFormat and a bunch of other classes built tight into your standard library aren't thread safe and now you need to go through your program and find out what else you missed. Go too has these fancy green threads at the cost of manually locking resources and finding out about race conditions when you forget about them.
Futures aren't a fundamental CS mistake, they're a design decision. You may disagree with that decision, but the advantage Rust brings is that you don't need to worry about thread safety once your program actually compiles, at the cost of different code styles.
Neither asynchronous processing design is fundamentally wrong, they both have their strengths and weaknesses.
> Virtual threading is fun and all until you find out SimpleDateFormat and a bunch of other classes built tight into your standard library aren't thread safe and now you need to go through your program and find out what else you missed. Go too has these fancy green threads at the cost of manually locking resources and finding out about race conditions when you forget about them.
Why would that ever be an issue? Instances of those classes shouldn't be shared between virtual threads just the same as when using regular threads.
Always with the blooming red and blue functions. You can say exactly the same thing about const.
The fact that a function can perform asynchronous operations matters to me and I want it reflected in the type system. I want to design my system on such a way that the asynchronous parts are kept where they belong, and I want the type system's help in doing that. "May perform asynchronous operations" is a property a calling function inherits from its callee and it is correctly modelled as such. I don't want to call functions that I don't know this about.
Now you can make an argument that you don't want to design your code this way and that's great if you have another way to think about it all that leads to code that can be maintained and reasoned about equally well (or more so). But calling the classes of functions red and blue and pretending the distinction has no more meaning than that is not such an argument. It's empty nonsense.
> The fact that a function can perform asynchronous operations matters to me and I want it reflected in the type system.
async doesn't tell you whether the function performs asynchronous operations, despite the name. async is an implementation detail about how the function must be invoked.
As TFA correctly points out, there's nothing stopping you from calling a blocking function inside a future, and blocking the whole runtime thread.
I didn't say it tells me whether the function does perform such operations, I said it tells me it can. More importantly it tells me which functions (most) can't.
It would be swell if functions could be generic over this capability at compile time, so that you could get the same guarantees from the type system without implementing the same protocols more than one time.
An async function is some syntactic sugar around a sync function that returns a future. You can merrily call one from the other.
You can only convert an int to a float with significant caveats. It's not a general trivial conversion. More complicated types may not be convertible at all or behave in all sorts of exciting ways (including having arbitrary side effects).
The point is that none of that is different to async functions. Of course you have to know what to do with them for them to be useful, but there is no requirement for them to "infect" calling code.
I want colored functions. I want to know which code is running synchronously and which doesn't, which raises errors and which doesn't. Color is just a description of the function's properties (and effects) and how it's compatible with other colors.
There is also nothing fundamentally bad with cooperative scheduling in scope of a single process.
I got into programming in the 1990s. At that point in time, there was still a large contingent of programmers loudly insisting they needed assembly language to do everything. And to be clear, I mean, everything. Not "Yeah, I can't really bring up an OS without a bit of specialized assembly" but "every programmer should write every program in assembly".
The vast majority of them were already wrong. They only got more wrong.
You may just be used to knowing what code is "synchronous" and what isn't because it's been shoved into your face and you've adapted your thought process to it. In practice, "everything important is doing something 'asynchronously'" turns out to be the vast majority of what you need, and the vast majority of your mental energy you are dedicated to splitting the world in two is a waste. For the little bit that remains, by all means use something specialized, but it's just not something that everyone, everywhere, needs to be doing all the time, any more than everyone everywhere should be manually allocating registers, or any more than programs need to have line numbers because otherwise how can they work? (One of my favorites because I remember having that conception myself.)
I think you have your analogy backwards: the "assembly programmer" in this situation is the person who doesn't understand why one would "color" functions and/or express a fundamental property as part of their types. "Why do we need to express this in their type? Every programmer should be able to understand this without help".
It's nonsense. Async in rust is just syntactic sugar around a function signature. You can merrily call async functions from sync rust, you just have to know what to do with the future you get back.
"you just have to know what to do" is the problem. You can call any color from any color, but for some colors it's trivial, e.g. sync function from a sync or async one, or a non-failing function from a failing or non-failing one.
I don't want to be able to call fallible function from an infallible one trivially, I want the compiler to force me to specify what exactly I'm going to do with an error if it happens. Likewise for async-from-sync: there are many ways I could call these: I can either create a single threaded executor and use it to complete the future to completion, or maybe I want to create a multithreaded executor, or maybe I expect the future complete in a single poll and never suspend and I don't even need a scheduler.
Well yes to all that. I still don't see the problem. An async function isn't really an async function, it's a sync function that returns a future. Would it be better if all that was manual? I've done quite a bit of stuff using manual async traits and it's painful and I highly value the syntax sugar that async brings. That said, I certainly don't want some executor running quietly behind the scenes doing async stuff for me without my explicit and full control. If I want to manually poll a future, that's for me to decide.
You seem to raise valid points and I don't disagree with you, however I don't see how it's relevant to the original concern regarding colored functions.
I suppose I'm struggling to understand what "colour" means in the context of Rust. It's surely just another word for signature. For some reason it's trotted out every time there's a discussion about async. I can only assume it's to do with the original use of the term for JavaScript async (which I know almost nothing about and have no opinion on), but I just cannot see its point in Rust async.
It has to do with the fact that most of the code in the project is not async but having to call async functions often propagates all the way to your main function. It's infectious and many people don't like it, myself included, that's why I'm working with Elixir and Golang where async is transparent and 99% automatic, or explicit but non-infectious, respectively.
I do love Rust and found a number of very valid uses for it but its async story leaves a lot to be desired. I don't enjoy writing it though I do enjoy the results.
> Without function coloring there's no distinction that you need to know of
Why do you think you don't need to know of it? I want to know if the function I'm calling is going to make a network request. Just because I can have a programming language that hides that distinction from me doesn't mean I want that.
Ideally I want to have the fundamental behavior of any function I call encoded in the function signature. So if it's async, I know it's going to reach out to some external system for some period of time.
getaddrinfo() is a synchronous function that can do network requests to resolve DNS. The network property isn't reflected in its function signature becoming async. You can have an async_getaddrinfo() which does, but the former is just a practical example of network calls in particular being unrelated to function coloring.
I think with stackful coroutines you lose low-overhead interoperability with C. Also, it possible to use stackless coroutines without introducing async/await 'colors'.
1. inability to read an async result from a sync function, which is a legitimately major architectural limitation.
2. author's opinion how function syntax should look like (fully implicit, hiding how the functions are run).
And from this there is the endless confusion and drama.
The problem 1 is mostly limited to JS. Languages that have threads can "change colour" of their functions at will, so they don't suffer from the dramatic problem described in the article.
But people see languages don't fit the opinion 2, of having magic implicit syntax, and treat it as an equally big deal the dead-end problem 1. But two syntaxes are somewhere between minor inconvenience to actual feature. In systems programming it's very important which type of locks you use, so you really need to know what runs async.
Maybe the issue is that we overload the concept of a function with an entirely different thing, a Future / Promise. Maybe if the syntax would have been entirely different too, it would have been easier to understand. We tend to have different syntax for different things.
I’m hesitant towards not distinguishing different things anymore and let the underlying system “figure it out”. I’m sure this could work as long as you’re on the happy path, but that’s not the only path there is.
620 comments
[ 3.5 ms ] story [ 348 ms ] threadThe lifetime of an Arc isn’t unknowable, it’s determined by where and how you hold it.
I think maybe the disconnect in this article is that the author is coming at Rust and trying to force their previous mental models on to it (such as garbage collection) rather than learning how to work with the language. It’s a common trap for anyone trying a new programming language, but Rust seems to trip people up more than most.
Obviously it's not random. It's statically unknowable.
In the same sense that the lifetime of an object in a GC'd system has a lower bound of, "as long as it's referenced", sure. But that's nearly the opposite of what the borrow checker tries to do by statically bounding objects, at compile time.
> maybe the disconnect in this article is that the author is coming at Rust and trying to force their previous mental models on to it
The opposite actually! I spent about a decade doing systems programming in C, C++, and Rust before writing a bunch of Haskell at my current job. The degree to which a big language runtime and GC weren't a boogeyman for some problem spaces was really eye-opening.
Rust will do a lot of invisible memory relocations under the covers. Which can work great in single threaded contexts. However, once you start talking about threading those invisible memory moves are a hazard. The moment shared memory comes into play everything just gets a whole lot harder with the rust async story.
Contrast that with a language like java or go. It's true that the compiler won't catch you when 2 threads access the same shared memory, but at the same time the mental burden around "Where is this in memory, how do I make sure it deallocates correctly, etc" just evaporates. A whole host of complex types are erased and the language simply cleans up stuff when nothing references it.
To me, it seems like GCs simply make a language better for concurrency. They generally solve a complex problem.
I don't think it's quite accurate to point to "invisible memory relocations" as the problem that pinning solves. In most cases, memory relocations in Rust are very explicit, by moving an owned value when it has no live references (if it has any references, the borrow checker will stop you), or calling mem::replace() or mem::swap(), or something along those lines.
Instead, the primary purpose of pinning is to mark these explicit relocations as unsafe for certain objects (that are referenced elsewhere by raw pointer), so that external users must promise not to relocate certain objects on pain of causing UB with your interface. In C/C++, or indeed in unsafe Rust, the same idea can be more trivially indicated by a comment such as /* Don't mess with this object until such-and-such other code is done using it! */. All pinning does is to enforce this rule at compile time for all safe code.
The new api lets you allocate "memory segments", which are byte arrays/C style structs. Such segments can be passed to native code easily or just used directly, deallocated with or without GC, bounds errors are blocked, use-after-free bugs are blocked, and segments can also be confined to a thread so races are also blocked (all at runtime though).
Unfortunately it only becomes available as a finalized non-preview API in Java 22, which is the release after the next one. In Java 21 it's available but behind a flag.
https://openjdk.org/jeps/8310626
Because when stackless coroutines run they don’t have access to the stack that existed when they were created. everything that used to be on the stack needs to get packaged up in a struct (this is what `async fn` does). However now everything that used to point to something else on the stack (which rust understands and is fine with) now points to something else within the “impl Future” struct. Hence you have self referential structs.
Basically the problem is that async blocks/fns/generators need to create a struct that holds all the local variables within them at any suspension/await/yield point. But local variables can contain references to other local variables, so there are parts of this struct that reference other parts of this struct. This creates two problems:
- once you create such self-references you can no longer move this struct. But moving a struct is safe, so you need some unsafe code that "promises" you this won't happen. `Pin` is a witness of such promise.
- in the memory model having an `&mut` reference to this struct means that it is the only way to access it. But this is no longer true for self referential structs, since there are other ways to access its contents, namely the fields corresponding to those local variables that reference other local variables. This is the problem that's still open.
Arc isn't an end-run around the borrow checker. If you need mutable references to the data inside of Arc, you still need to use something like a Mutex or Atomic types as appropriate.
> The degree to which a big language runtime and GC weren't a boogeyman for some problem spaces was really eye-opening.
I have the opposite experience, actually. I was an early adopter of Go and championed Garbage Collection for a long time. Then as our Go platforms scaled, we spent increasing amounts of our time playing games to appease the garbage collector, minimize allocations, and otherwise shape the code to be kind to the garbage collector.
The Go GC situation has improved continuously over the years, but it's still common to see libraries compete to reduce allocations and add complexity like pools specifically to minimize GC burden.
It was great when we were small, but as the GC became a bigger part of our performance narrative it started to feel like a burden to constantly be structuring things in a way to appease the garbage collector. With Rust it's nice to be able to handle things more explicitly and, importantly, without having to explain to newcomers to the codebase why we made a lot of decisions to appease the GC that appear unnecessarily complex at first glance.
The reason for that is the compiler quality, the design tradeoffs and Go's GC implementation throughput are simply not there for it to ever be a good general purpose systems-programming-oriented language.
Go receives undeserved hype, for use cases C# and Java are much better at due to their superior GC implementations and codegen quality (with C# offering better lower level features like structs+generics and first-class C interop).
But yes, GC is very much not free and is an explicit tradeoff vs compile time + manual memory management.
Making stong statements without a backup in hard facts is a sign of zealotry...
For example, you may be interested in this read: https://blog.twitch.tv/en/2019/04/10/go-memory-ballast-how-i...
Issues like these simply don't happen with GCs in modern JVM implementations or .NET (not saying they are perfect or don't have other shortcomings, but the sheer amount of developer hours invested in tuning and optimizing them far outstrips Go).
I don’t see how running into an OOM problem is necessarily a problem with the GC. That said, Java is a memory intensive language, it’s a trade off that Java is pretty up front about.
I don’t have a horse in this race but I would be quite surprised if Go’s GC implementation could even hold a candle to the ones found in C# and Java. They have spent literally decades of research and development, and god knows how much money (likely north of $1b), optimizing and refining their GC implementations. Go just simply lacks any of the sort of maturity and investment those languages have.
Since the advent of Java in mid-90s I hear about superiority of its VM, yet my observations from the ops PoV claim otherwise. So I suspect a huge hoax...
Hey btw, you're saying "Java is _memory intensive_", like it would magically explain everything. Let's get to that more deeply. Why is it so, dear Watson? Have you compared the memory consumption of the same algo and pretty much similar data structures between languages? Why Java has to be such a memory hog? Why also its class loading is so slow? Are these a qualities of superior VM design and zillions of man-hour invested? huh?
By the way, if the code implementing functionality X needs N times more memory than the other language with gc, then however advanced that gc would be (need to find a proof for that btw), it wouldn't catch up speedwise, because it simply needs to move around more. So simple.
Marketing is not a silver bullet for success and the tech industry is full of examples of exactly that. The truth is that Sun was able to promote Java so heavily because it was found to be useful.
Since the advent of Java in mid-90s I hear about superiority of its VM, yet my observations from the ops PoV claim otherwise.
The landscape of the 90s certainly made a VM language appealing. And compared to the options of that day it's hardly any wonder.
So I suspect a huge hoax...
It's you verses a plurality, if not majority, of the entire enterprise software market. Of course that's not to say that Java doesn't have problems or that the JVM is perfect, but is it so hard to believe that Java got something right? Is it honestly more believable that everyone else is caught up in a collective delusion?
Hey btw, you're saying "Java is _memory intensive_", like it would magically explain everything.
It's not that Java is necessarily memory intensive, but that a lot of Java performance tuning is focused towards optimizing throughput performance, not memory utilization. Cleaning out a large heap occasionally is in general better than cleaning out a smaller one more frequently.
By the way, if the code implementing functionality X needs N times more memory than the other language with gc, then however advanced that gc would be (need to find a proof for that btw), it wouldn't catch up speedwise, because it simply needs to move around more. So simple
It's not so simple. First of all, the choice of a large heap is not mandated by Java, it's a trade off that developers are making. Second of all, GC performance issues only manifest when code is generating a lot of garbage, and believe it or not, Java can be written to vastly minimize the garbage produced. And last of all, Java GCs like Shenandoah have a max GC pause time of less than 1ms for heaps up to 16TiB.
Anyway, at the end of the day no one is going to take Go away from you. Personally I don't have a horse in this race. That said, the fact is that Java GCs are far more configurable, sophisticated, and advanced than anything Go has (and likely ever will). IMO, Go came at a point in time where there was a niche to exploit, but that niche is shrinking.
But I think you are avoiding a direct answer to the question why Java needs so much memory in the first place. You say about "developer's choice for a big heap", first I don't think it is their choice, but the consequence of the fact that such a big heap is needed at all, for a typical code. Why?
Let's code a basic https endpoint using typical popular framework returning some simple json data. Usually stuff. Why it will be consuming 5x - 10x more memory for Java? And, if one says it's just unrealistic microbenchmark, things go worse when coding more real stuff.
Btw,having more knobs for a gc is not necessarily a good thing, if it means that there are no fire-and-forget good defaults. If an engineer needs continously to get his head around these knobs to have a non-crashing app, then we have problem. Or rather - ops have a problem, and some programmers are, unfortunately, disconnected from the ops realm. Have you been working together with ops guys? On prod, ofc?
These are not the same.
The problem with GC'd systems is that you don't know when the GC will run and eat up your cpu cycles. It is impossible to determine when the memory will actually be freed in such systems. With ARC, you know exactly when you will release your last reference and that's when the resource is freed up.
In terms of performance, ARC offers massive benefits because the memory that's being dereferenced is already in the cache. It's hard to understate how big of a deal this is. There's a reason people like ARC and stay away from GC when performance actually begins to matter. :)
It's more like "you notice when it happens". You don't know in advance when the last reference will be released (if you did, there would be no point in using reference counting).
> In terms of performance, ARC offers massive benefits because the memory that's being dereferenced is already in the cache.
It all depends on your access patterns. When ARC adjusts the reference counter, the object is invalidated in all other threads' caches. If this happens with high frequency, the cache misses absolutely demolish performance. GC simply does not have this problem.
> There's a reason people like ARC and stay away from GC when performance actually begins to matter.
If you're using a language without GC built in, you usually don't have a choice. When performance really begins to matter, people reach for things like hazard pointers.
A barista knows when a customer will pay for coffee (after they have placed their order). A barista does not know when that customer will walk in through the door.
> (if you did, there would be no point in using reference counting).
There’s a difference between being able to deduce when the last reference is dropped (for example, by profiling code) and not being able to tell anything about when something will happen.
A particular developer may not know when the last reference to an object is dropped, but they can find out. Nobody can guess when GC will come and take your cycles away.
> The cache misses absolutely demolish performance
With safe Rust, you shouldn’t be able to access memory that has been freed up. So cache misses on memory that has been released is not a problem in a language that prevents use-after-free bugs :)
> If you’re using a language without GC built in, you usually don’t have a choice.
I’m pretty sure the choice of using Rust was made precisely because GC isn’t a thing (in all places that love and use rust that is)
Sorry, no chance of deciphering that.
> There’s a difference between being able to deduce when the last reference is dropped (for example, by profiling code) and not being able to tell anything about when something will happen.
> A particular developer may not know when the last reference to an object is dropped, but they can find out.
The developer can figure out when the last reference to the object is dropped in that particular execution of the program, but not in the general sense, not anymore than they can in a GC'd language.
The only instance where they can point to a place in the code and with certainty say "the reference counted object that was created over there is always destroyed at this line" is in cases where reference counting was not needed in the first place.
> With safe Rust, you shouldn’t be able to access memory that has been freed up. So cache misses on memory that has been released is not a problem in a language that prevents use-after-free bugs :)
I'm not sure why you're talking about freed memory.
Say that thread A is looking at a reference-counted object. Thread B looks at the same object, and modifies the object's reference counter as part of doing this (to ensure that the object stays alive). By doing so, thread B has invalidated thread A's cache. Thread A has to spend time reloading its cache line the next time it accesses the object.
This is a performance issue that's inherent to reference counting.
> I’m pretty sure the choice of using Rust was made precisely because GC isn’t a thing (in all places that love and use rust that is)
Wanting to avoid "GC everywhere", yes. But Rust/C++ programs can have parts that would be better served by (tracing) garbage collection, but where they have to make do with reference counting, because garbage collection is not available.
but it also has big disadvantage, that it communicates to actual malloc for memory management, which is usually much less performant than GC from various reasons.
Can you elaborate?
I've seen a couple of malloc implementations, and in all of them, free() is a cheap operation. It usually involves setting a bit somewhere and potentially merging with an adjacent free block if available/appropriate.
malloc() is the expensive call, but I don't see how a GC system can get around the same costs for similar reasons.
What am I missing?
I've seen some benchmarks, but can't find them now, so maybe I am wrong about this.
> free() is a cheap operation. It usually involves setting a bit somewhere and potentially merging with an adjacent free block if available/appropriate.
there is some tree like structure somewhere, which then would allow to locate this block for "malloc()", this structure has to be modified in parallel by many concurrent threads, which likely will need some locks, meaning program operates outside of CPU cache.
In JVM for example, GC is integrated into thread models, so they can have heap per thread, and also "free()" happens asynchronously, so doesn't block calling code. Additionally, malloc approaches usually suffer from memory fragmentation, while JVM GC is doing compactions all the time in background, tracks memory blocks generations, and many other optimizations.
- A moving (and ideally, generational) GC means that you can recompact the heap, making malloc() little more than a pointer bump.
- This also suggests subsequent allocations will have good locality, helping cache performance.
Manual memory management isn't magically pause-free, you just get to express some opinion about where you take the pauses. And I'll contend that (A) most programmers aren't especially good at choosing when that should be, and (B) lots (most?) software cares about overall throughput, so long as max latency stays under some sane bound.
In many cases this means it's much cheaper than objects in languages with implicit reference counting.
The point about wrangling with Weak suggests that they're trying to build complex ownership structures (which, to be fair, would be easier in to deal with a single thread) which isn't really something easy to express in Rust in general. I use weak smart pointers exceedingly rarely. Outside of the first section (which isn't talking about async Rust specifically, it's just speaking about concurrency generally) channels aren't even mentioned. They're the main thing I use for communication between different parts of my program when writing async code and when interfacing between async and non-async code, plus the other signalling abstractions like Notify, semaphores, etc. Mutexes are slow and bottlenecky and shared state quickly gets complicated to manage, this has been known for ages. I think the problem might be more the `BIG_GLOBAL_STATIC_REF_OR_SIMILAR_HORROR` in the first place.
The comment about nothing stopping you from calling blocking code in an async context is valid, but it's relatively manageable and you can use `tokio::spawn_blocking` or similar when you must do it.
I think it's a fair assumption to say that the author is aware of what Arcs are and how they work. I believe their point is more so that because of how async works in Rust, users have to reach for Arc over normal RAII far more often than in sync code. So at a certain point, if you have a program where 90% of objects are refcounted, you might as well use a tracing GC and not have the overhead of many small heap allocations/frees plus atomic ops.
Perhaps there are in fact ways around Arc-ing things for the author's use cases. But in my (limited) experience with Rust async I've definitely run into things like this, and plenty of example code out there seems to do the same thing [1].
For what it's worth, I've definitely wondered whether a real tracing GC (e.g. [2]) could meaningfully speed up many common async applications like HTTP servers. I'd assume that other async use cases like embedded state machines would likely have pretty different performance characteristics, though.
[0] https://en.wikipedia.org/wiki/Garbage_collection_(computer_s...
[1] https://tokio.rs/tokio/tutorial/shared-state
[2] https://manishearth.github.io/blog/2015/09/01/designing-a-gc...
Fair, but when reading an article like this I have to refer to what's written, not what we think the author knew but didn't write.
…on a server where you can have a ton of RAM. It's superior on client machines because it's friendlier to swapped out memory, which is why Swift doesn't have a GC.
I'm currently plumbing through some logic to call a sync method on a struct that implements Future and it's... an interesting challenge.
While we can make zero-cost async abstractions somewhat easy for users, the library developers are the ones who suffer the pain.
A good candidate for this is Graal. It can compile (JIT/AOT) both WASM and also LLVM bitcode directly so Rust programs can have full hardware/OS access without WASM limitations, and in theory it could allow apps to fully benefit from the work done on Loom and async. The pieces are all there. The main issue is you need to virtualize IO so that it goes back into the JVM, so the JVM controls all the code on the stack at all times. I think Graal can do this but only in the enterprise edition. Then you'd be able to run ~millions of Rust threads.
[0] https://lunatic.solutions/
[1] https://wasmcloud.com
You cannot run scoped fibers, forcing you to "Arc shit up", Pins are unusable without unsafe, and a tiniest change in an async-function could make the future !Send across the entire codebase.
What I'm missing at the end of the article is the author's point: I believe they're advocating for the use of raw threads and manual management of concurrency, and doing away with the async paraphernalia. But, at the same time, earlier in the article they give the example of networking-related tasks as something that isn't so easy to deal with using only raw threads.
So, taking into account that await&co. are basically syntactic sugar + an API standard (iirc, I haven't used Rust so much lately), I wonder about what the alternative is. In particular, it seems to me like the alternative you could have would be everyone rolling their own "concurrency API", where each crate (inconsistently) exposes some sort of `await()` function, and you have to manually roll your async runtime every time. This would obviously also not be ideal.
> Maybe Rust isn’t a good tool for massively concurrent, userspace software. We can save it for the 99% of our projects that don’t have to be.
Personally, I'm a bit more radical than the author. You won't be able to write software like the example correctly. It should just not be done, ever. Machines can still optimize some sanely organized software into the same thing, maybe, if it happens to be a tractable problem (I'm not sure anybody knows). But people shouldn't touch that thing.
I have some quibbles with this article:
"Rust comes at this problem with an “async/await” model"
No, it does not. It allows for that, and there's a big ... community ... around the async stuff, but in reality the language is entirely fine with operating using explicit concurrency constructs. And in fact for most applications I think standard synchronous functions combined with communicating channels is cleaner. I work in code bases that do both, and I find the explicit approach easier to reason about.
In the end, Async is something people ideally reach for only after they hit the wall with blocking on I/O. But in reality they're often reaching for it just because -- either because it's cool... or because some framework they are relying on mandates it.
But I think the pendulum will swing back the other way at some point. I don't think it's fair to tar the whole language with it.
This is like saying C++ allows for templates, and theres a big community around it. Sure, but its the entire community.
Believe it or not, there's other types of things being built in Rust. Systems work, which I think Rust is more appropriate for.
Maybe async is the most popular concurrency construct there (I have no idea). But the entire population here is small.
I don't think it's "the entire community" at all. Dealing with futures across library calls is a pain and almost every library that can avoid it, will avoid it.
I try to avoid async code because of its annoying pain points and I rarely see any circumstances where spawning a new thread doesn't work. Sure, there's more overhead, and you need some kind of limiting factor to prevent spawning a billion of them, but async isn't really required in most circumstances.
It's like saying Go allows for generics. Very few people and libraries bother with them. Working with them is kind of a pain. They're there jf you want to use them, but you generally don't.
The guy's got a point in that doing a bunch of Arc, RwLock, and general sharing of state is going to get messy. Especially once you are sprinkling 'static all over the place, it infects everything, much like colored functions. I did this whole thing once back when I was starting off where I would Arc<RwLock> stuff, and try to be smart about borrow lifetimes. Total mess.
But then rust also has channels. When you read about it, it talks about "messages", which to me means little objects. Like a few bytes little. This is the solution, pretty much everything I write now is just a few tasks that service some channels. They look at what's arrived and if there's something to output, they will put a message on the appropriate channel for another task to deal with. No sharing objects or anything. If there's a large object that more than one task needs, either you put it in a task that sends messages containing the relevant query result, or you let each task construct its own copy from the stream of messages.
And yet I see a heck of a lot of articles about how to Arc or what to do about lifetimes. They seem to be things that the language needs, especially if you are implementing the async runtime, but I don't understand why the average library user needs to focus so much on this.
But green threads were not and are not the right solution for Rust, so it's kind of beside the point. Async Rust is difficult, but it will eventually be possible to use Async Rust inside the Linux kernel, which is something you can't do with the Go approach.
Rust has changed a lot in the past 5 years, people just haven't noticed, so they assume that Rust is still an old outdated language.
Async is just hard. That’s it. It’s fundamentally difficult.
In my experience language implementations of async fall into 2-axes: clarity and control. C# is straightforward-enough (having cribbed its async design off functional languages) but I find it scores low on the “clarity” scale and moderate-high in control, because you could control it, but it was t always clear.
JS is moderate-high clarity, low control: easy to understand, because all the knobs are set for you. Before it got async/await sugar, I’d have said it would have been low clarity, because I’ve seen the promise/callback hell people wrote when given rope.
Python is the bottom of the barrel for both clarity and control. It genuinely has to have the most awful and confusing async design I’ve ever seen.
I personally find Rust scores high in both clarity and control. Playing with the Glommio executor was what really solidified my understanding of how async works however.
What I realized, eventually, is that blocking is a beautiful thing. Embrace the thread of execution going to sleep, as another thread may now execute on the (single core at the time) CPU.
Now you have an organization problem, how to distribute threads across different tasks, some sequential, some parallel, some blocking, some nonblocking. Thread-per-request? Thread-per-connection?
And now a management problem. Spawning threads. Killing threads. Thread pools. Multithreaded logging. Exceptions and error handling.
Totally manageable in mild cases, and big wins in throughput, but scaling limits will present themselves.
I confront many of these tradeoffs in a fun little exercise I call "Miner Mover", implemented in Ruby using many different concurrency primitives here: https://github.com/rickhull/miner_mover
Basically Rust Futures is what Go wishes it could have. Rust made the right choice in waiting and spending the time to design async right.
Colored functions is a debatable problem at best. I consider it a feature not a bug and it makes reasoning about programs easier at the expense of writing additional async/await keywords which is really a very minor annoyance.
On the other hand Go's need of using channels to do trivial and common tasks like communicating the result of an async task together with the lack of RAII and proper cleanup signaling in channels (you can very easily deadlock if nothing is attached on the other end of the channel), plus no compile time race detection - all that makes writing concurrent code harder.
Rust: it turns out that not every concurrency needs to be zero-cost abstraction
(But if you're only firing up a few tasks, why not just use threads? To get a nice wrapper around an I/O event loop?)
It really annoys me that something like this isn't built-in: https://github.com/mrkline/channel-drain
We could imagine extending this to arbitrary poll-able things. And now we have futures, kind of.
I fully acknowledge that I'm an "old school" system dev who's coming from the C world and not the JS world, so I probably have a certain bias because of that, but I genuinely can't understand how anybody could look at the mess that's Rust's async and think that it was a good design for a language that already had the reputation of being very complicated to write.
I tried to get it, I really did, but my god what a massive mess that is. And it contaminates everything it touches, too. I really love Rust and I do most of my coding in it these days, but every time I encounter async-heavy Rust code my jaw clenches and my vision blurs.
At least my clunky select "runtime" code can be safely contained in a couple functions while the rest of the code remains blissfully unaware of the magic going on under the hood.
Dear people coming from the JS world: give system threads and channels a try. I swear that a lot of the time it's vastly simpler and more elegant. There are very, very few practical problems where async is clearly superior (although plenty where it's arguably superior).
In the mean time it is a little annoying to use, but I don’t mind designing against it by default. I feel less architecturally constrained if more syntactically constrained.
I've used Rust async extensively for years, and I consider it to be the cleanest and most well designed async system out of any language (and yes, I have used many languages besides Rust).
I agree that being able to use `async` inside of traits would be very useful, and hopefully we will get it soon.
> generally needing more capability to existentially quantify Future types without penalty
Could you clarify what you mean by that? Both `impl Future` and `dyn Future` exist, do they not work for your use case?
> Async function types are a mess to write out.
Are you talking about this?
Or this? > More control over heap allocations in async/await futures (we currently have to Box/Pin more often than necessary).I'm curious about your code that needs to extensively Box. In my experience Boxing is normally just done 1 time when spawning the Future.
> Async drop.
That would be useful, but I wouldn't call the lack of it "half-baked", since no other mainstream language has it either. It's just a nice-to-have.
> Better cancellation.
What do you mean by that? All Futures/Streams/etc. support cancellation out of the box, it's just automatic with all Futures/Streams.
If you want really explicit control you can use something like `abortable`, which gives you an AbortHandle, and then you can call `handle.abort()`
Rust has some of the best cancellation support out of any async language I've used.
> Async iteration.
Nicer syntax for Streams would be cool, but the combinators do a good job already, and StreamExt already has a similar API as Iterator.
Golang supports running asynchronous code in defers, similar with Zig when it still had async.
Async-drop gets upgraded from a nice-to-have into an efficiency concern as the current scheme of "finish your cancellation in Drop" doesn't support borrowed memory in completion-based APIs like Windows IOCP, Linux io_uring, etc. You have to resort to managed/owned memory to make it work in safe Rust which adds unnecessary inefficiency. The other alternatives are blocking in Drop or some language feature to statically guarantee a Future isn't cancelled once started/initially polled.
So does Rust. You can run async code inside `drop`.
And this would be basically what you have to do in Go anyways - you need to explicitly use defer if you want code to run on destruction, with the caveat that in Go nothing stops you from forgetting to call it, when in Rust I can at least have a deterministic guard that would panic if I forget to call the explicit destructor before the object getting out of scope.
BTW async drop is being worked on in Rust, so in the future this minor annoyance will be gone
It'd be very nice to be able to use `impl` in more locations, representing a type which needs not be known to the user but is constant. This is a common occurrence and may let us write code like `fn foo(f: impl Fn() -> impl Future)` or maybe even eventually syntax sugar like `fn foo(f: impl async Fn())` which would be ideal.
Re: Boxing
I find that a common technique needed to get make abstraction around futures to work is the need to Box::pin things regularly. This isn't always an issue, but it's frequent enough that it's annoying. Moreover, it's not strictly necessary given knowledge of the future type, it's again more of a matter of Rust's minimal existential types.
Re: async drop and cancellation.
It's not always possible to have good guarantees about the cleanup of resources in async contexts. You can use abort, but that will just cause the the next yield point to not return and then the Drops to run. So now you're reliant on Drops working. I usually build in a "kind" shutdown with a timer before aborting in light of this.
C# has a version of this with their CancelationTokens. They're possible to get wrong and it's easy to fail to cancel promptly, but by convention it's also easy to pass a cancelation request and let tasks do resource cleanup before dying.
Re: Async iteration
Nicer syntax is definitely the thing. Futures without async/await also could just be done with combinators, but at the same time it wasn't popular or easy until the syntax was in place. I think there's a lot of leverage in getting good syntax and exploring the space of streams more fully.
> I fully acknowledge that I'm an "old school" system dev who's coming from the C world and not the JS world, so I probably have a certain bias because of that, but I genuinely can't understand how anybody could look at the mess that's Rust's async and think that it was a good design for a language that already had the reputation of being very complicated to write.
I'm in the same "old school" system dev category as you, and I think that modern languages have gone off the deep end, and I complained about async specifically in a recent comment on HN: https://news.ycombinator.com/item?id=37342711
> At least my clunky select "runtime" code can be safely contained in a couple functions while the rest of the code remains blissfully unaware of the magic going on under the hood.
And we could have had that for async as well, if languages were designed by the in-the-trenches industry developer, and not the "I think Haskell and Ocaml is great readability" academic crowd.
With async in particular, the most common implementation is to color the functions by qualifying the specific function as async, which IMO is exactly the wrong way to do it.
The correct way would be for the caller to mark a specific call as async.
IOW, which of the following is clearer to the reader at the point where `foo` is called?
Option 1: color the function
Option 2: schedule any function Option 1 results in compilation errors for code in the call-stack that isn't async, results in needing two different functions (a wrapper for sync execution), and means that async only works for that specific function. Option 2 is more like how humans think - schedule this for later execution, when I'm done with my current job I'll wait for you if you haven't finished.What if your example code is holding onto a thread that foo() is waiting to use?
Said another way, explain how you solved the problems of just synchronously waiting for async. If that just worked then we wouldn't need to proliferate the async/await through the stack.
Why? It isn't solved for async functions, is it? Just because the async is propagated up the call-stack doesn't mean that the call can't deadlock, does it?
Deadlocks aren't solved for a purely synchronous callstack either - A grabbing a resource, then calling B which calls C which calls A ...
Deadlocks are potentially there whether or not you mix sync/async. All that colored functions will get you is the ability to ignore the deadlock because that entire call-stack is stuck.
> If that just worked then we wouldn't need to proliferate the async/await through the stack.
It's why I called it a leaky abstraction.
Maybe I'm misunderstanding what you are saying. I use the word "_implementation_type_" below to mean "either implemented as option 1 or option 2 from my post above."
With current asynchronous implementations (like JS, Rust, etc), any time you use `await` or similar, that statement may never return due to a deadlock in the callstack (A is awaiting B which is awaiting C which is awaiting A).
And if you never `await`, then deadlocking is irrelevant to the _implementation_type_ anyway.
So I am trying to understand what you mean by "it cannot deadlock in this way" - in what way do you mean? async functions can accidentally await on each other without knowing it, which is the deadlock I am talking about.
I think I might understand better if you gave me an example call-chain that, in option 1, sidesteps the deadlock, and in option 2, deadlocks.
A is sychrounously waiting B which is awaiting C which could complete but never gets scheduled because A is holding onto the only thread. Its a very common situation when you mix sync and async and you're working in a single threaded context, like UI programming with async. Of course it can also cause starvation and deadlock in a multithreaded context as well but the single thread makes the pitfall obvious.
That's specifically why I called it a Leaky Abstraction in my first post on this: too many people are confusing a particular implementation of asynchronous function calls with the concept of asynchronous function calls.
I'm complaining about how the mainstream languages have implemented async function calls, and how poorly they have done so. Pointing out problems with their implementation doesn't make me rethink my position.
Besides Javascript, its also a common problem in C# when you force synchronous execution of an async Task. I'm fairly sure its a problem in any language that would allow an async call to wait for a thread that could be waiting for it.
I really can't imagine how your proposed syntax could work unless the synchronous calls could be pre-empted, in which case, why even have async/await at all?
But I look forward to your implementation.
It's an implementation issue, because "running on only a single thread" is an artificial constraint imposed by the implementation. There is nothing in the concept of async functions, coroutines, etc that has the constraint "must run on the same thread as the sync waiting call".
An "abstraction" isn't really one when it requires knowledge of a particular implementation. Async in JS, Rust, C#, etc all require that the programmer knows how many threads are running at a given time (namely, you need to know that there is only one thread).
> But I look forward to your implementation.
Thank you :-)[1]. I actually am working (when I get the time, here and there) on a language for grug-brained developers like myself.
One implementation of "async without colored functions" I am considering is simply executing all async calls for a particular host thread on a separate dedicated thread that only ever schedules async functions for that host thread. This sidesteps your issue and makes colored functions pointless.
This is one possible way to sidestep the specific example deadlock you brought up. There's probably more.
[1] I'm working on a charitable interpretation of your words, i.e. you really would look forward to an implementation that sidesteps the issues I am whining about.
Actually, Rust could still learn a lot from these languages. In Haskell, one declares the call site as async, rather than the function. OCaml 5 effect handlers would be an especially good fit for Rust and solve the "colouration" problem.
Rust adopted the stackless coroutine model for async tasks based on its constraints, such as having a minimal runtime by default, not requiring heap allocations left and right, and being amenable to aggressive optimizations such as inlining. The function coloring problem ("contamination") is an unfortunate consequence. The Rust devs are currently working on an effects system to fix this. Missing features such as standard async traits, async functions in traits, and executor-agnosticism are also valid complaints. Considering Rust's strict backwards compatibility guarantee, some of these will take a long time.
I like to think of Rust's "async story" as a good analogue to Rust's "story" in general. The Rust devs work hard to deliver backwards compatible, efficient, performant features at the cost of programmer comfort (ballooning complexity, edge cases that don't compile, etc.) and compile time, mainly. Of course, they try to resolve the regressions too, but there's only so much that can be done after the fact. Those are just the tradeoffs the Rust language embodies, and at this point I don't expect anything more or less. I like Rust too, but there are many reasons others may not. The still-developing ecosystem is a prominent one.
Anyway, while I have some issues with async around futur composition and closures, I see people with the kind of super strong reaction here and just feel like I must not be seeing something. To me, it solves the job well, is comprehensible and relatively easy to work with, and remains performant at scale without too much fiddling.
(This is assuming you are already switching to communicating using channels or similar abstraction.)
Most often when doing async you have a small number of tasks repeated many times, then you spin up one thread per CPU, and "randomly" assign each task as it comes in to a thread.
When doing GUI style programming you have a lot of different tasks and each task is done in exactly one thread.
Any software that does a lot of fine-grained concurrent I/O has this issue. Database engines have been fighting this for many years, since they can pervasively block both on I/O and locking for data model concurrency control.
"Async" in native code is cargo cult, unless you're trying to run on bare metal without OS support.
There is a reason why so many developers have chosen to do application level scheduling: No operating system has exposed viable async primitives to build this on the OS level. OS threads suck so everyone reinvents the wheel. See Java's "virtual threads", Go's goroutines, Erlang's processes, NodeJS async.
You don't seem to be aware what a context switch on an application level is. It is often as simple as a function call. There is no way that returning to the OS, running a generic scheduler that is supposed to deal with any possible application workload that needs to store all the registers and possibly flush the TLB if the OS makes the mistake of executing a different process first and then restore all the registers can be faster than simply calling the next function in the same address space.
Developers of these systems brag about how you can have millions of tasks active at the same time without breaking any sweat.
[1]: https://news.ycombinator.com/item?id=36790238
To get easier timers, to make cancellation at all possible (how to cancel a sync I/O operation?), and to write composable code.
There are patterns that become simpler in async code and much more complicated in sync code.
And since the cancellation logic runs on the cancellable thread, you can't really cancel a blocking operation. What you can do is to let it run to completion, check that it was canceled, and discard the value.
The cancellation thread is generally the one doing the `select` so it spawns the operation thread(s) and waits for (one of) their results (i.e. through a channel/event). The others which lose the race are sent the cancellation signal and optionally joined if they need to be (i.e. they use intrusive memory).
Hoare's later paper introduced buffered channels to CSP.
So one can use it as synchronous passing, or queued passing.
As a wise programmer once said, "Do not communicate by sharing memory; instead, share memory by communicating"
I imagine Rust to be a language far more similar to Go, in both use cases and functionality, than JS.
I mostly agree. But I would wager that for a significant amount of people their first exposure to "async" is JS and not any number of other languages. And when you try to write async Rust the same way as you might write async JS, things just aren't that pretty.
It's great, but message passing it is not.
If you haven't seen this paper, I bet you'll find at least one or two new bugs that you didn't know about: https://songlh.github.io/paper/go-study.pdf
In most languages, current for loop value is always accessed a variable, not a reference. The only languages where it's not the case that I know of are Go and Python (JavaScript used to also have this problem with for(var ...), it was fixed with for(let ...)). So if you don't regularly write Go, it's easy to make this mistake.
Rust gives you channels (both synchronous blocking channels and async channels), and they work great, there is nothing stopping you from using them.
I write a fair amount of code in Elixir professionally and this isn't how I view it.
There are some specific Elixir/Erlang bits of ceremony you need to do to set up your supervision tree of GenServers, but then once that's done you get to write code that feels like so gle threaded "ignore the rest of the world" code. Some of the function calls you're making might be "send and message and wait for a response" from GenServers etc. but the framework takes care of that.
I wrote some driver code for an NXP tag chip. Driving the inventory process is a bit involved, you have to do a series of things, set up hardware, turn on radio, wait a bit, send data, service the SPI the whole time in parallel. With the right setup for the hardware interface I just wrote the whole thing as a sequence, it was the simplest possible code you could imagine for it. And this at the same time as running a web server, and servicing hardware interrupts that cause it to reload the state of some registers and show them to each connected web session.
When you can guarantee sole ownership, why not put that exclusive pointer in the message? I’d think that this sort of compile-time lock would be an important advantage for the type system. (I think some VMs actually do this sort of thing dynamically, but I can’t quite remember where I read about it.)
On a multiprocessor, there’s of course a balance to be determined between the overhead of shuffling the object’s data back and forth between CPUs and the overhead of serializing and shuffling the queries and responses to the object’s owning thread. But I don’t think the latter approach always wins, does it? At least I can’t tell why it obviously should.
When moving between threads I do what you suggest here and use channels to send signals rather than having a lot of shared state. Sometimes there is a crucial global state something that’s easier to just directly access, but I just write struct that manages all the Arc/RwLock or whatever other exclusive access mechanism I need for the access patterns. From the callers point of view everything is just a simple function call. When writing the struct I need to be thoughtful of sharing semantics but it’s a very small struct and I write it once and move on.
I also don’t understand their concern about making things Send+Sync. In my experience almost everything is easily Send+Sync, and things that aren’t shouldn’t or couldn’t be.
I get that sometimes you just want to wear sweatpants and write code without thought of the details, but most languages that offer that out of the box don’t really offer efficient concurrency and parallelism. And frankly you rarely actually need those things even if the “but it’s cool” itch is driving you. Most of the time a nodejs-esque single threaded async program is entirely sufficient, and a lot of the time Async isn’t even necessary or particularly useful. But when you need all these things, you probably need to hike up your sweatpants and write some actual computer code - because microseconds matter, profiled throughput is crucial, and nothing in life that’s complex is easy and anyone selling you otherwise is lying.
This is a recurring pattern I've started to notice with Rust: most things that repeatedly feel clunky, or noisy, or arduous, can be wrapped in an abstraction that allows your business logic to come back into focus. I've started to think this mentality is essential to any significant Rust project.
I think I’m used to other languages provided a lot of these abstractions or having some framework that manages it all. The frameworks in rust tend to be pretty low level (with a few notable exceptions) so perhaps that’s where it comes from.
Another thing, for me, was that I came from mostly writing TypeScript, which is the opposite: the base language is breezy without abstractions, and the type system equips you to strongly-type plain data and language features, so you'll have a great time if you stick to those
But yeah, it's been interesting to see how different the answers to these questions can be in different languages!
That makes abstractions far more useful and powerful, since you never need to do a cost-benefit analysis in your head, abstractions are just always a good idea in Rust.
So I assume you mean "implementation complexity" but that's irrelevant, because that cost only needs to be paid once, and then you put the abstraction into a crate, and then millions of people can benefit from that abstraction.
No abstraction is perfect. Every abstraction, when encountered by a user, requires them to ask "what does this do?", because they don't have the implementation in front of their eyes
This may be an easy question to answer- maybe it maps very obviously to a pattern or domain concept they already know, or maybe they've seen this exact abstraction before and just have to recall it
It may be slightly harder- a new but well-documented concept, or a concept that's intuitive but complex, or a concept that's simple but poorly-named
Or it may be very hard- a badly-designed abstraction, or one that's impossible to understand without understanding the entire system
But the simplest, most elegant, most intuitive abstraction in the world has nonzero cognitive cost. We abstract despite the cost, when that cost is smaller than the cost of not abstracting.
Abstractions allow systems to scale. Without them, it would be impossible to work on a system that's 1M lines of code long, because you'd have to read and understand all 1M lines before doing anything.
The "zero-cost" phrase is deceptive. There's a non-zero cognitive cost to the author and all subsequent readers. A proliferation of abstractions increases the cost of every other abstraction further due to complex interactions. This is true of in all languages where the community has embraced the idea of abstraction without moderation.
Rust embraces zero to low cost abstraction at the machine performance level, although to get reflective or runtime adaptive abstractions you end up losing some of that zero cost as you need to start boxing and moving things into heaps and using vtables, etc. IMO this is where rust is weakest and most complex.
No, the cognitive cost of a particular abstraction relative to all other abstractions under consideration can be negative.
The option of not using any abstraction doesn’t exist. If you disagree with that then I think we have to go back one step and ask what an abstraction even is.
These can also be quite narrow: Rc is a zero-cost abstraction for refcounting with both strong and weak references allocated with the object on the heap. You cannot implement something the same more efficiently, but you can implement something different but similar that is both faster and lighter than Rc. You can make a CheapRc that only has strong counts, and that will be both lighter and faster by a tiny amount, or a SeparateRc that stores the counts separately on the heap, which offers cheaper conversions to/from Rc.
We're talking about the comparison between using an abstraction vs not using an abstraction.
When I said "doesn't have a runtime cost", I meant "the abstraction doesn't have a runtime cost compared to not using the abstraction".
If you want your computer to do anything useful, then you have to write code, and that code has a runtime cost.
That runtime cost is unavoidable, it is a simple necessity of the computer doing useful work, regardless of whether you use an abstraction or not.
Whenever you create or use an abstraction, you do a cost-benefit analysis in your head: "does this abstraction provide enough value to justify the EXTRA cost of the abstraction?"
But if there is no extra cost, then the abstraction is free, it is truly zero cost, because the code needed to be written no matter what, and the abstraction is the same speed as not using the abstraction. So there is no cost-benefit analysis, because the abstraction is always worth it.
>Rust embraces abstractions because Rust abstractions are zero-cost. So you can liberally create them and use them without paying a runtime cost.
>you never need to do a cost-benefit analysis in your head, abstractions are just always a good idea in Rust
Again though, and ignoring that, "zero-cost abstraction" can be very narrow and context specific, so you really don't need to go out of your way to find "costly" abstractions in Rust. As an example, if you have any uses of Rc that don't use weak references, then Rc is not zero-cost for those uses. This is rarely something to bother about, but rarely is not never, and it's going to be more common the more abstractions you roll yourself.
Async the keyword doesn’t, but Tokio forces all of your async functions to be multi thread safe. And at the moment, tokio is almost exclusively the only async runtime used today. 95% of async libraries only support tokio. So you’re basically forced to write multi thread safe code even if you’d benefit more from a single thread event loop.
Rust async’s set up is horrid and I wish the community would pivot away to something else like Project Loom.
https://docs.rs/tokio/1.32.0/tokio/task/fn.spawn_local.html
There is a lot of misinformation in this thread, with people not knowing what they're talking about.
Every executor (including tokio) provides a `spawn_local` function that spawns Futures on the current thread, so they don't need to be Send:
https://docs.rs/tokio/1.32.0/tokio/task/fn.spawn_local.html
I have used Rust async extensively, and it works great. I consider Rust's Future system to be superior to JS Promises.
> I have used Rust async extensively, and it works great. I consider Rust's Future system to be superior to JS Promises.
Sure, but it’s a major headache compared to Java VirtualThreads or goroutines
No your not, you spawn a runtime on each thread and use spawn_local on each runtime. This is how actix-web works and it uses tokio under the hood.
https://docs.rs/actix-rt/latest/actix_rt/
thread_local! exists, and you can just call spawn_local on each thread. You can even call spawn_local multiple times on the same thread if you want.
You can have some parts of your programs be multi-threaded, and then other parts of your program can be single-threaded, and the single-threaded and multi-threaded parts can communicate with an async channel...
Rust gives you an exquisite amount of control over your programs, you are not "stuck" or "locked in", you have the flexibility to structure your code however you want, and do async however you want.
You just have to uphold the basic Rust guarantees (no data races, no memory corruption, no undefined behavior, etc.)
The abstractions in Rust are designed to always uphold those guarantees, so it's very easy to do.
It does.
Problem is that there isn't the documentation, examples etc to help navigate the many options.
I am genuinely asking because I have little formal background in CS so "runtimes" and actual low level differences between , for instance, async and green threads mystifies me. EG What makes them actually different from the "runtime" perspective?
While there are other runtimes that are always single-threaded, you can do it with tokio too. You can use a single threaded tokio runtimes and !Send tasks with LocalSet and spawn_local. There are a few rough edges, and the runtime internally uses atomics where a from-the-ground-up single threaded runtime wouldn't need them, but it works perfectly fine and I use single threaded tokio event loops in my programs because the tokio ecosystem is broader.
The dream of Smalltalk and true OOP is still alive.
If you say Smalltalk is better OOP I might agree, but calling it "true" is not correct.
I consider the definition e.g. used by IEEE as sufficiently strict, see e.g. https://ethw.org/Milestones:Object-Oriented_Programming,_196..., but - as you say - it's not the defintion used by Kay.
There's also a written conversation you can find online where he disqualifies pretty much all of the mainstream languages of being OO.
A lot of people, like you, say that OO == ADTs. Or rather, what ever Simula, C++ and Java are doing. Some will say that inheritance is an integral part of it, other's say it's all about interfaces.
But then there's people who say that Scheme and JavaScript are more object oriented than Java and C#. Or that when we're using channels or actors we're now _really_ doing OOP.
There's people who talk about patterns, SOLID, clean code and all sorts of things that you should be adhering to when structuring OO code.
Then there's people who say that OO is all about the mental model of the user and their ability to understand your program in terms of operational semantics. They should be able to understand it to a degree that they can manipulate and extend it themselves.
It's all very confusing.
This is pretty unlikely. See https://news.ycombinator.com/item?id=36879311.
That's publications though. Alan Kay says he used it in conversation in 1967: http://userpage.fu-berlin.de/~ram/pub/pub_jf47ht81Ht/doc_kay...
There's probably also a distinction to be made between "object-oriented" and "object-oriented programming".
Right, including Smalltalk 76 and 80 onwards themselves. Remember Kay's statement "actually I made up the term object-oriented and I can tell you I did not have C++ in mind, so the important thing here is I have many of the same feelings about Smalltalk" (https://www.youtube.com/watch?v=oKg1hTOQXoY&t=636s); the reason he refers to Smalltalk this way in his 1997 talk was likely the fact that Smalltalk-80 has more in common with Simula 67 than his brain child Smalltalk-72. Ingalls explicitly refers to Simula 67 in his 2020 HOPL paper.
> and reject the things that make Smalltalk different
Which would mostly be its dynamic nature (Smalltalk-76 can be called the first dynamic OO language) and the use of runtime constructs instead dedicated syntax for conditions and loops (as it is e.g. the case in Lisp). There are a lot of dynamic OO languages still in use today, e.g. Python. Also Smalltalk-80 descendants are still in use, e.g. Pharo.
When you need the absolute best performance sharing state is sometimes better - but you need a deep understanding of how your CPUs share state. A mutex or atomic write operation is almost always needed (the exceptions are really weird), and those will kill performance so you better spend a lot of time minimizing where you have them.
I would also suggest looking into ringbuffers and LMAX Disruptor pattern.
There is also Red Planet Lab's Rama, which takes the data flow idea and uses it to scale.
https://doc.rust-lang.org/nomicon/races.html
Ultimately, it depends on your data model.
From https://news.ycombinator.com/item?id=37289579 :
> I haven't checked, but by the end of the day, I doubt eBPF is much slower than select() on a pipe()?
Channels have a per-platform implementation.
- "Patterns of Distributed Systems (2022)" (2023) https://news.ycombinator.com/item?id=36504073
Async code can scale essentially infinitely, because it can multiplex thousands of Futures onto a single thread. And you can have millions of Futures multiplexed onto a dozen threads.
This makes async ideal for situations where your program needs to handle a lot of simultaneous I/O operations... such as a web server:
http://aturon.github.io/blog/2016/08/11/futures/
Async wasn't invented for the fun of it, it was invented to solve practical real world problems.
like “send request to channel A with message 123, make sure to get a response back from channel B exactly for that message”
The async patterns in Rust, especially with regards to data safety assurances for the compiler, are emblematic of this philosophy. Though there are complexities, the value proposition is a safer concurrency model that requires developers to think deeply about their data and execution flow. I do concur that Rust might not be the go-to for every massively concurrent userspace application, but for systems where robustness and safety are paramount, the trade-offs are justifiable. It's also worth noting that as the ecosystem evolves, we'll likely see more abstractions and libraries that ease these pain points.
Still, diving into the intricacies as this article does, gives developers a better foundational understanding, which in itself is invaluable.
Async functions don't have to always own their arguments. Just the outermost future that is getting spawned on another thread has to. The rest of the async program can borrow arguments as usual. You don't need to spawn() every task — there are other primitives for running multiple futures, with borrowed data, on the same thread.
In fact, this ability for a future to borrow from itself is the reason why Rust has native await instead of using callbacks. Futures can be "self-referential" in Rust, and nothing else is allowed to.
Other times however rust stops me from writing buggy code and where I didn’t quite understand what I was doing. In some sense it can help you understand what your software better (when the problem isn’t an implementation detail).
I get the authors frustration, I often have the same feelings. Sometimes you just want to tell rust to get out of your way.
As an aside, I think there is room for a language similar to golang with sum types and modules and be a joy.
I enjoy Rust, and I love how the compiler helps me solve problems. However, the ecosystem is "async or gtfo", or "just write it yourself if you dont want async lmao", and that's not good enough.
Right now even building a library that support multiple async runtimes is a PITA, I have done it a couple times. So you end up supporting either just tokio and maybe async-std.
https://docs.rs/futures/latest/futures/executor/fn.block_on....
imagine you have an:
you can: but this does get messy if the async code you're running isn't runtime-agnostic :(article says you can panic if you use the pattern you show. specifically, if you call `my_normal_code()` from an async context.
is the author just talking about a quirk in tokio? or is this sort of wrapping intrinsically dangerous somehow?
This is one of the goals of the async working group. Hopefully, when ready, that'll make it possible to swap out async runtimes underneath arbitrary code without issues.
Async/await was a terrible idea for fixing JavaScript's lack of proper blocked threading that is currently being bolted onto every language. It splits every language and every library-ecosystem in half and will cause pains for many years to come.
Everyone who worked with multi-threading outside of JavaScript knows that using actors/communicating sequential processes is the best way to do multi-threading.
I recently found an explanation for that in Joe Armstrong's thesis. He argues that the only way to understand multi-threaded programs is writing strictly sequential code for every thread and not muddling all the code for all the threads in one place:
"The structure of the program should exactly follow the structure of the problem. Each real world concurrent activity should be mapped onto exactly one concurrent process in our programming language. If there is a 1:1 mapping of the problem onto the program we say that the program is isomorphic to the problem.
It is extremely important that the mapping is exactly 1:1. The reason for this is that it minimizes the conceptual gap between the problem and the solution. If this mapping is not 1:1 the program will quickly degenerate, and become difficult to understand. This degeneration is often observed when non-CO languages ["non concurrency-oriented", looking at you JavaScript!] are used to solve concurrent problems. Often the only way to get the program to work is to force several independent activities to be controlled by the same language thread or process. This leads to a inevitable loss of clarity, and makes the programs subject to complex and irreproducible interference errors." [0]
[0] https://erlang.org/download/armstrong_thesis_2003.pdf
There is also a good rant against async/await by Ron Pressler who implemented project loom in java: https://www.youtube.com/watch?v=oNnITaBseYQ
As fun as it is to hate on JavaScript, it's really interesting to go back and watch Ryan Dahl's talk introducing Node.js to the world (https://www.youtube.com/watch?v=EeYvFl7li9E). He's pretty ambivalent about it being JavaScript. His main goal was to find an abstraction around the epoll() I/O event loop that didn't make him want to tear his eyes out, and he tried a bunch of other stuff first.
Agreed. JavaScript was actually my first language after TurboPascal in 1996.
I was also there listening to the first podcasts when node came out.
JavaScript is a very interesting language, especially with it's prototype memory model. And the eventloop apart from the language is interesting as well. And it's no coincidence Apple went as far as baking optimizations for JavaScript primitive operations into the M1 microcode.
But I still think multithreading is best done by using blocking operations.
NIO can be implemented on top of blocking IO as far as I know but not the other way round.
Also, sidenote, I think JavaScript's only real failure is the lack of a canonical module/import system. That error lead to countless re-implementations of buildsystems and tens of thousands of hours wasted debugging.
Agreed. I don't hate on JS, in fact I think it's the best tool for the several very common use cases it targets, and I'll even defend the way objects work in it (i.e. lets me do what I want with minimal fuss). The import/require drama was annoying, though.
I don't think it's a "good" solution in the abstract, but in the concrete of "I have a dynamically-typed scripting language with already over a decade of development and many more years of development that will happen before the event-based stuff is really standard", it's nearly the only choice. Python's gevent was the only other thing I saw that kinda solved the problem, and I really liked it, but I'm not sure it's a sustainable model in the end as it involves writing a package that aggressively reaches into other packages to do its magic; it is a constant game of catch-up.
I do think it's a grave error in the 2020s to adopt async as the only model for a language, though. There are better choices. And I actually exclude Rust here, because async is not mandatory and not the only model; I think in some sense the community is making the error of not realizing that your task will never have more than maybe a hundred threads in it and a 2023 computer will chomp on that without you noticing. Don't scale for millions of concurrent tasks when you're only looking at a couple dozen max, no matter what language or environment you're in. Very common problem for programmers this decade. It may well be the most impactful premature optimization in programming I see today.
And also with fibers/virtual threads (project loom) you can actually have a million threads using blocking hand-off on one machine. So the performance argument is kind of gone.
but I get it, you can always go back to the promises and callbacks if you want.
JS callbacks are indeed better than C callbacks because you can hold onto some state. Although I guess the capture is implicit rather than explicit, so some people might say it's more confusing.
I'm pretty sure Joyent adopted and funded node.js because they were doing lots of async code in C, and they liked the callback style in JavaScript better. It does match the kind of problems that Go is now being used for, and this was pre-Go.
But anyway it is interesting how nobody really talks about callbacks anymore. Seems like async/await has taken over in most languages, although I sorta agree with the parent that it could have been better if designed from scratch.
I actually think it was a great solution in JS/TS given it's a single threaded event loop. The lower level the language the worse of an abstraction it is though. So I think most of the complaints here about async Rust are valid.
It was used because of ineptitude of languages where it become popular, and its far easier to implement into GC-less languages than message-passing-based asynchronous, but it's just misery to write code in. I'd prefer to suffer Go ineptitudes just to use bastardised message passing called channels there rather than any of the Python/JS/Rust async.
It was created to be an improvement over the Javascript situation, and somehow every language that had a sane structure adopted it as if it was not only good, but the way to do things. This is insane.
On the voodoo ridden land that is software development, we have plenty of clearly harmful fads that are much older than Rust and yet practiced everywhere.
Up to now, rust async has lasted for less time than the NoSQL craziness. I'm hard pressed to think of any large fad that lasted less than it.
Btw, the turnaround time is longer with a database, which often forms the foundation of a system. NoSQL bandwagoning was so destructive in part because of how long it looked like a good idea each time. Same with ORMs.
I see this repeated everywhere in this thread. async/await originated in C# not JS.
- the C# implementation predates even Promises in JS, so it is not "the same implementation" and your implication that C# was inspired by JS as opposed to the other way around is false. More background: [0]
- Typescript works fine with the JS implementation so any differences aren't for type safety reasons, but largely because C# has a multithreaded event loop unlike JS
Also promises (or "futures" as they're called elsewhere) aren't unique to any language. They're used in lots of places that predate both C# and JS's use, for example the twisted framework in Python.
[0]: https://news.ycombinator.com/item?id=37438486
For language designers I point out in my other comment that Anders authored both C# and TS. TS' influence on ES6 is documented publicly.. heck TC39 has an open proposal to add type annotations to JS now!
As for users, any dev that touches frontend has to write JS, unless you're purely a mobile or desktop shop (even then, there's electron). So yes, I think tons of folks willingly write C# and JS/TS. I'm certainly one (though write more Python than both these days). Was I an early adopter of async in Python because of my familiarity of it in C#/JS? You bet I was. Maybe I'm "insane."
can you tell why it is not how not to do it in your opinion? What are the obvious issues with this approach?
JVM's futures are a joy to work with compared to JS's promises (or Kotlin's coroutines for that matter). While similar, I don't think you can conflate them.
Sure, Rust is certainly verbose and very strict how the ownership rules apply in the context of async, but this is a hard constraint of its memory safety model. We could probably do better while retaining all performance but this is by far one of the best implementations. Another example of nice to use async/await is C# which trades performance/memory (state has to be boxed if it is to live across continuations) for convenience (you just write it naturally without worrying about underlying behavior).
There is a reason Rust toyed with "green threads" at its inception but decided against such. The only popular languages of today that do these are Go and Java (which basically forced to do this because you can't go async without introducing the feature early in the lifecycle of the language, and the authors of project Loom are simply wrong with their excuses why this is superior to async/await).
Async/await is here to stay and is the right abstraction, git good, and it's not even difficult to use anyway.
[0] where feature name is green threads, not doing concurrency at all, doing it manually, etc.
I guess I should specify that this is true even in a single-threaded context and even without any additional buffers or whatever.
I just write the same code that I would write if it were synchronous and it executes asynchronously.
var user = service.GetUser(id);
var promos = service.GetPromotions(category);
var eligible = GetEligibility(await user, await promos);
The point of async/await is that it converts your function into a reentrant state machine (in a different way than compiling a sync function already turned it into a state machine.) The problem with the usual design is that it uses futures, which are bad because they have dynamic lifetimes.
Why?
It's probably the right abstraction for Haskell, or any other language that works well with functional programming, lambdas and monads. Loom is a better fit for Java. Rust also would have probably been better off with something else. Effect handlers might have been a good choice.
Maybe in the 2000's but I feel this reasoning is no longer valid in 2023 and should be put to rest.
10k problem.. Wouldn't modern computing not work if my Linux box couldn't spin up 10k threads? Htop says I'm currently at 4,000 threads on an 8 core machine.
By the 2010s the problem had been updated to C10M. The people discussing it (well, perhaps some) aren't idiots and understand that the threshold changes as hardware changes.
Also, the issue isn't creating 10k threads it's dealing with 10k concurrent users (or, again, a much higher number today).
The async case is suited to situations where you're blocking for things like network requests. In that case the thread will be doing nothing, so we want to hand off the work to another task of some kind that is active. Green threads mean you can do that without a context switch.
I have no doubt that having a thread per core and managing the data with only non-blocking operations is much faster. But I'm pretty current machines can manage a thousand or so threads locked almost the entire time just fine.
I'm not sure how that's relevant here, if for example something takes 1ms and I do it 1000 times a second, I'm using 1000 ms of CPU time vs not doing it at all. So if you want to use big o notation in this context it should be O(n) where n is the number of context switches, because you are not comparing algorithms used to switch between threads but you are comparing doing context switch or not doing it at all.
It got even more expensive in recent years after all the speculative execution vulnerabilities in CPUs, so now you have additional logic on every context switch with mitigations on in kernel.
> The numbers reported here paint an interesting picture on the state of Linux multi-threaded performance in 2018. I would say that the limits still exist - running a million threads is probably not going to make sense; however, the limits have definitely shifted since the past, and a lot of folklore from the early 2000s doesn't apply today. On a beefy multi-core machine with lots of RAM we can easily run 10,000 threads in a single process today, in production. As I've mentioned above, it's highly recommended to watch Google's talk on fibers; through careful tuning of the kernel (and setting smaller default stacks) Google is able to run an order of magnitude more threads in parallel.
So do we discard existing ways of making software more efficient because we can be more wasteful on more recent hardware? What if we could develop our software such that 2000s computers are still useful, rather than letting those computers become e-waste?
I think having to keyword async is frustrating as a design decision
* While the author states that not many apps "need" high concurrency in userspace... I would invert that and say that we may be missing so much performance, new potential applications, etc because highly concurrent code is so hard to get right. One bit of evidence of this (to me at least) is how often in my career I have had to scale things up due to memory or other resource limitations and not CPU. And when it is CPU, so often looking into it more finds bugs with concurrency that are the root cause or at least exacerbate the issue
* While I completely agree that rust is not easy with async and have myself poked around at which magical type things I need to do each time I have touched async rust code, I don't really like the suggestion being to "go use a different language", first, because if you are picking up rust, you (IMHO) should have a very good reason to already have chosen it. Rust is not easy enough or ubiquitous enough that you should be choosing it "just for fun" and your reason for using Rust should be compelling enough that you (right now) are willing to put in the effort to learn async when you need it
* What the other mentions in the body of the article, but I think is more of what my suggestion would be: don't use async unless you need it!. While I would love to see Rust (and think it should) evolve to the point where async is "easy", maybe we instead just need to get more pragmatic in what is taught and written about. I think when people start Rust they want to use all the fanciness, which includes async, and while some of that is just programmers, I think it is also how tutorials, docs, and general communication about a programming language happens where we show the breadth of capability, rather than the more realistic learning path, which leads people to feel like if they don't use async, they aren't doing it right
Finally, I do really hope Rust keeps working on the promise of these zero cost abstractions that can really simplify things... but if that doesn't work, I am at least hopeful of what people can build on top of the rust featureset/toolchain to help make things like async more realistic to be the default without the need for a complex VM/runtime.
I know parallel ATA cables were all the rage. They had a higher theoretical throughput when compared with serial ATA cables but there was too much cross-talk involved to make it actually faster in the end so now we have serial ATA cables everywhere with much higher throughput than parallel ATA cables could ever achieve.
Should we move back away from parallelism and focus on handling synchronous stuff faster instead?
I want to write stuff to disk (SSD these days). I can issue a request, then have to wait tens to hundreds of milliseconds (in the average case, the worst case can be far longer) for that request to finish and let me know that my I/O request succeeded or failed. There's no getting around that with present-day technology.
The situation is worse and even less reliable with network I/O. If you are talking to a server in another continent, the speed of light determines the minimum of time I hear back from it, even if it (and all the intermediary network links) are lightly loaded and functioning perfectly.
> Should we move back away from parallelism and focus on handling synchronous stuff faster instead?
Rust already has excellent handling of synchronous computation, given that it can meet/sometimes exceed equivalent performance in C. The problem is when you're I/O or network bound; you can either throw threads at the problem (and by extension throw memory at the problem for the thread stacks) or use async programming.
Typically, if you want to build something with Rust, it'll have to use async, at least because gRPC and the like are implemented that way. So the vanilla (and excellent, IMO) Rust language doesn't exist there. Everything is async from the get-go.
A weird way to use Rust since you can do a lot of messaging within the process, and use the computing power much more efficiently.
RPC is essentially messaging and message-passing. Message-passing is a way to avoid mutable shared state - this is the model with which Go became successful.
RPC surely has its use but message passing is another, and very often inferior, solution to the problem set where Rust has excellent own solutions for.
Isn't that already, in this strong generality, an almost always wrong assumption?
Sure, one can do massively parallel or embarrassingly parallel computation.
Sure, graphic cards are parallel computers.
Sure, OS kernels use multiple cores.
Sure, languages and concepts like Clojure exist and work - for a specific domain, like web services (and for that, Clojure works fascinatingly well).
But there are many, even conceptually simple algorithms which are not easy to parallelize. There is no efficient parallel Fast Fourier Transform I know of.
I haven't investigated it deeply, but I was developing something in Rust, and whether something needs to be threadsafe or not is entirely on the consumer's use case... bad separation of concerns for the provider of a generic interface to have to specify the specific type of boxed value. 100% fine if the behavior in this case is to pre-allocate the max possible boxed type memory requirement.
This is the only thing I was really frustrated with in Rust
Your generic interface just takes a reference to the value inside the box.
Using Arc everywhere solves it, but dumb and inefficient for non threaded use cases. Maybe compiler optimizes this though, who knows. Semantically it's wrong though.
Honestly forget the specifics enough at this point to discuss so I'll drop it haha.
Was just curious whether somebody else was tracking this, or there was a known workaround. I think it's something the language will eventually support. I saw other threads on rustlang asking for the same thing, and best I saw was some sort of enum style hack representing the boxed types to emulate it
If it's dynamic, you can use Cow or the supercow/bos/... crates if you want Arc/Rc to be options as well.
I'll check the Cow crate.
Until all the work you're trying to push is generating so many allocations that your GC goes to shit once every two minutes trying to clean up the mess you made. (https://discord.com/blog/why-discord-is-switching-from-go-to...)
Not sure where does it stand on a global competition rating board (if there's even such a thing) but it's pretty good. I've never seen it crap the bed, though I also never worked at the scale of Twitch and Discord.
If you have a service that handles massive amounts of network calls at the core (think linkerd, nginx, etc.), or you want to have a massive amount of lightweight tasks in your game, or working on an embedded software where you want cooperative concurrency, async Rust is an amazing super-power.
Most system/application level things is not going to need async IO. Your REST app is going to be perfectly fine with a threadpool. Even when you do need async, you probably want to use it in a relatively small part of your software (network), while doing most of the things in threads, using channels to pass work around between async/blocking IO parts (aka hybrid model).
Rust community just mindlessly over-did using async literally everywhere, to the point where the blocking IO Rust (the actually better UX one) became a second class citizen in the ecosystem.
Especially visible with web frameworks where there is N well designed async web frameworks (Axum, Wrap, etc.) and if you want a blocking one you get:
But it also praises Go for its implementation, which is also based on a coroutine of a different kind. Stackful coroutines, which do not have any of these problems.
Rust considered using those (and, at first, that was the project's direction). Ultimately, they went to the stackless operation model because stackfull coroutine requires a runtime that preempts coroutines (to do essentially what the kernel does with threads). This was deemed too expensive.
Most people forget, however, that almost no one is using runtime-free async Rust. Most people use Tokio, which is a runtime that does essentially everything the runtime they were trying to avoid building would have done.
So we are left in a situation where most people using async Rust have the worst of both worlds.
That being said, you can use async Rust without an async runtime (or rather, an extremely rudimentary one with extremely low overhead). People in the embedded world do. But they are few, and even they often are unconvinced by async Rust for their own reasons.
I think that stackless coroutines are better than stackfull, in particular for Rust. Everything was done correctly by the Rust team.
Again, this is all fair and good, as long as people understand the tradeoff and make good technical decisions around. If they all jump on async bandwagon blind o the obvious limitations, we get where Rust ecosystem is now.
Stackful coroutines don't require a preemptive runtime. I certainly hope that we didn't end up with colored functions in Rust because of such a misconception.
This is used to keep track of task runtime quotas so they can yield as soon as possible afterward.
This is the same technique used in Go and many others for preemption. If you don't add this, futures that don't yield can run forever, stalling the system.
You are right that it is not strictly necessary, but in practice, it is so helpful as a guard against the yielding problem that it's ubiquitous.
> I certainly hope that we didn't end up with colored functions in Rust because of such a misconception.
Misconceptions are everywhere unfortunately!
This is honestly shocking to hear. I would think that if people had bugs in their programs they would want them to fail loudly so they can be fixed.
Folks would rather have every future time sliced so that other tasks get some CPU time in a ~fair way (after all, there is no concept of task priority in most runtime).
But you're right: it isn't required, and you could sprinkle every loop of your code with yielding statements. But knowing when to yield is impossible for a future. If nothing else is running, it shouldn't yield. If many things are running but the problem space of the future is small, it probably shouldn't yield either, etc.
You simply do not have the necessary information in your future to make an informed decision. You need some global entity to keep track of everything and either yield for you or tell you when you should yield. Tokio does the former, Glommio does the latter.
It gets even more complex when you add IO into the mix because you need to submit IO requests in a way that saturates the network/nvme drives/whatever. So if a future submits an IO request, it's probably advantageous to yield immediately afterward so that other futures may do so as well. That's how you maximize throughput. But as I said, that's a very hard problem to solve.
I guess if someone wants to use futures as if they were goroutines then it's not a bug, but this sort of presupposes that an opinionated runtime is already shooting signals at itself. Fundamentally the language gives you a primitive for switching execution between one context and another, and the premise of the program is probably that execution will switch back pretty quickly from work related to any single task.
I read the blog about this situation at https://tokio.rs/blog/2020-04-preemption which is equally baffling. The described problem cannot even happen in the "runtime" I'm currently using because io_uring won't just completely stop responding to other kinds of sqe's and only give you responses to a multishot accept when a lot of connections are coming in. I strongly suspect equivalent results are achievable with epoll.
So just like any other kind of scheduling? "Frequently" is also very subjective, and there are tradeoffs between throughput, latency, and especially tail latency. You can improve throughput and minimum latency by never preempting tasks, but it's bad for average, median, and tail latency when longer tasks starve others, otherwise SCHED_FIFO would be the default for Linux.
>I read the blog about this situation at https://tokio.rs/blog/2020-04-preemption which is equally baffling
You've misunderstood the problem somehow. There is definitely nothing about tokio (which uses epoll on Linux and can use io_uring) not responding in there. io_uring and epoll have nothing to do with it and can't avoid the problem: the problem is with code that can make progress and doesn't need to poll for anything. The problem isn't unique to Rust either, and it's going to exist in any cooperative multitasking system: if you rely on tasks to yield by themselves, some won't.
Yes. Industries that care about latency take some pains to avoid this as well, of course.
> io_uring and epoll have nothing to do with it and can't avoid the problem: the problem is with code that can make progress and doesn't need to poll for anything.
They totally can though? If I write the exact same code that is called out as problematic in the post, my non-preemptive runtime will run a variety of tasks while non-preemptive tokio is claimed to run only one. This is because my `accept` method would either submit an "accept sqe" to io_uring and swap to the runtime or do nothing and swap to the runtime (in the case of a multishot accept). Then the runtime would continue processing all cqes in order received, not *only* the `accept` cqes. The tokio `accept` method and event loop could also avoid starving other tasks if the `accept` method was guaranteed to poll at least some portion of the time and all ready handlers from one poll were guaranteed to be called before polling again.
This sort of design solves the problem for any case of "My task that is performing I/O through my runtime is starving my other tasks." The remaining tasks that can starve other tasks are those that perform I/O by bypassing the runtime and those that spend a long time performing computations with no I/O. The former thing sounds like self-sabotage by the user, but unfortunately the latter thing probably requires the user to spend some effort on designing their program.
> The problem isn't unique to Rust either, and it's going to exist in any cooperative multitasking system: if you rely on tasks to yield by themselves, some won't.
If we leave the obvious defects in our software, we will continue running software with obvious defects in it, yes.
Yeah, there's your misunderstanding, you've got it backwards. The problem being described occurs when I/O isn't happening because it isn't needed, there isn't a problem when I/O does need to happen.
Think of buffered reading of a file, maybe a small one that fully fits into the buffer, and reading it one byte at a time. Reading the first byte will block and go through epoll/io_uring/kqueue to fill the buffer and other tasks can run, but subsequent calls won't and they can return immediately without ever needing to touch the poller. Or maybe it's waiting on a channel in a loop, but the producer of that channel pushed more content onto it before the consumer was done so no blocking is needed.
You can solve this by never writing tasks that can take "a lot" of time, or "continue", whatever that means, but that's pretty inefficient in its own right. If my theoretical file reading task is explicitly yielding to the runtime on every byte by calling yield(), it is going to be very slow. You're not going to go through io_uring for every single byte of a file individually when running "while next_byte = async_read_next_byte(file) {}" code in any language if you have heap memory available to buffer it.
I assumed that users would issue reads of like megabytes at a time and usually receive less. Does the example of reading from a socket in the blog post presuppose a gigabyte-sized buffer? It sounds like a bigger problem with the program is the per-connection memory overhead in that case.
The proposal is obviously not to yield 1 million times before returning a 1 meg buffer or to call read(2) passing a buffer length of 1, is this trolling? The proposal is also not some imaginary pie-in-the-sky idea; it's currently trading millions of dollars of derivatives daily on a single thread.
>I'm not familiar with tokio so I did not know that it maintained buffers in userspace
Most async runtimes are going to do buffering on some level, for efficiency if nothing else. It's not strictly required but you've had an unusual experience if you've never seen buffering.
>filled them before the user called read()
Where did you get this idea? Since you seem to be quick to accuse others of it, this does seem like trolling. At the very least it's completely out of nowhere.
>it could still have read() yield and return the contents of the buffer.
If I call a read_one_byte, read_line, or read(N) method and it returns past the end of the requested content that would be a problem.
>I assumed that users would issue reads of like megabytes at a time and usually receive less.
Reading from a channel is the other easy example, if files were hard to follow. The channel read might implemented as a quick atomic check to see if something is available and consume it, only yielding to the runtime if it needs to wait. If a producer on the other end is producing things faster than the consumer can consume them, the consuming task will never yield. You can implement a channel read method that always yields, but again, that'd be slow.
>The proposal is obviously not to yield 1 million times before returning a 1 meg buffer, is this trolling
No, giving a illustrative example is not trolling, even if I kept the numbers simple to make it easy to follow. But your flailing about with the idea of requiring gigabyte sized buffers probably is.
Maybe you can read the linked post again? The problem in the example in the post is that data keeps coming from the network. If you were to strace the program, you would see it calling read(2) repeatedly. The runtime chooses to starve all other tasks as long as these reads return more than 0 bytes. This is obviously not the only option available.
I apologize for charitably assuming that you were correct in the rest of my reply and attempting to fill in the necessary circumstances which would have made you correct
This is just mundane non-blocking sockets. If the socket never needs to block, it won't yield. Why go through epoll/uring unless it returns EWOULDBLOCK?
> But I'm glad you're at least on the same page now, about how checking if something is ready and yielding to the runtime are separate things.
I haven't ever said otherwise?
You may be referring to this particular issue in Go https://github.com/golang/go/issues/10958 which I think was somewhat addresses a couple releases back.
https://tokio.rs/blog/2020-04-preemption#a-note-on-blocking
> Tokio does not, and will not attempt to detect blocking tasks and automatically compensate
I've used stackful coroutines many times in many codebases. It never required or used a runtime or preemption. I'm not sure why having a runtime that preempts them would even be useful, since it defeats the reason most people use stackful coroutines in the first place.
Can you tell us which? Go, Haskell and the other usual suspect all have runtime with automatic, transparent preemption.
Python also comes with 2 features that seem to be stackless coroutines with attached syntax ceremonies, but one of those 2 features is commonly used with a hefty runtime instead of being used for control flow. JavaScript comes with 2 features named similarly to those of Python, but only one of them seems to be "runtime-free" stackless coroutines.
If coroutines can be preempted then it introduces a requirement for concurrency control that otherwise doesn't need to exist and interferes with dynamic cache locality optimizations. These are some of the primary benefits of using stackful coroutines in this context.
Being able to interrupt a stackful coroutine has utility for dealing with an extremely slow or stuck thread but you want this to be zero-overhead unless the thread is actually stuck. In most system designs, the time required to traverse any pair of sequential yield points is well-bounded so things getting "stuck" is usually a bug.
Letting end-users inject arbitrary code into these paths at runtime does require the ability to interrupt the thread but even that is often handled explicitly by more nuanced means than random preemption. Sometimes "extremely slow" is correct and expected behavior, so you have to schedule around it.
However, async Rust is not using stackless coroutines for this reason - it's using stackless coroutines because they achieve a better performance profile than stackful coroutines. You can read all about it on Aaron Turon's blog from 2016, when the futures library was first released:
http://aturon.github.io/blog/2016/08/11/futures/
http://aturon.github.io/blog/2016/09/07/futures-design/
It is not the case that people using async Rust are getting the "worst of both worlds." They are getting better performance by default and far greater control over their runtime than they would be using a stackful coroutine feature like Go provides. The trade off is that it's a lot more complicated and has a bunch of additional moving parts they have to learn about and understand. There's no free lunch.
The other day, Intel revealed a processor with 66 thread support per core. 64 of those threads were called "slow", because there's no prefetching and speculative execution, as they are supposed to be waiting (mainly for memory, but networking could be another option). Perhaps very many cheap hardware threads is a way out of this.
I ask as someone who uses java and is about to rewrite a bunch of code to be able to chuck the entire async paradigm into the trash can and use a blocking model but on virtual threads where blocking is ok.
Could you expand a bit? Why?
[1] https://www.youtube.com/watch?v=WsCJYQDPrrE
Yes. I just noticed that Tokio was pulled into my program as a dependency. Again. It's not being used, but I'm using a crate which has a function I'm not using which imports reqwest, which imports h2, which imports tokio.
Please make it happen! I want my userspace software to be in Rust!
Although, if it won't happen, then even better, a free real estate for a RustScript.
Concurrency's correct primitive is Hoare's Communicating Sequential Processes mapped onto green threads. Some languages that have it right are Java (since JDK17 - Java Virtual Threads), Go, Kotlin.
Futures aren't a fundamental CS mistake, they're a design decision. You may disagree with that decision, but the advantage Rust brings is that you don't need to worry about thread safety once your program actually compiles, at the cost of different code styles.
Neither asynchronous processing design is fundamentally wrong, they both have their strengths and weaknesses.
true, but DateTimeFormatter has been available since Java 8, released almost 10 years ago.
VirtualThreads will be available in Java tomorrow
Also: https://docs.oracle.com/javase/8/docs/api/java/text/SimpleDa...
Why would that ever be an issue? Instances of those classes shouldn't be shared between virtual threads just the same as when using regular threads.
The fact that a function can perform asynchronous operations matters to me and I want it reflected in the type system. I want to design my system on such a way that the asynchronous parts are kept where they belong, and I want the type system's help in doing that. "May perform asynchronous operations" is a property a calling function inherits from its callee and it is correctly modelled as such. I don't want to call functions that I don't know this about.
Now you can make an argument that you don't want to design your code this way and that's great if you have another way to think about it all that leads to code that can be maintained and reasoned about equally well (or more so). But calling the classes of functions red and blue and pretending the distinction has no more meaning than that is not such an argument. It's empty nonsense.
"We" don't all agree on this.
async doesn't tell you whether the function performs asynchronous operations, despite the name. async is an implementation detail about how the function must be invoked.
As TFA correctly points out, there's nothing stopping you from calling a blocking function inside a future, and blocking the whole runtime thread.
you cannot convert an function that calls async code into a sync function.
u32 / float does not have the problem. It does not "bubble up", unless you want it to.
You can only convert an int to a float with significant caveats. It's not a general trivial conversion. More complicated types may not be convertible at all or behave in all sorts of exciting ways (including having arbitrary side effects).
The point is that none of that is different to async functions. Of course you have to know what to do with them for them to be useful, but there is no requirement for them to "infect" calling code.
Either way, all of these changes are really annoying to make. We want less of these annoyances, not more.
There is also nothing fundamentally bad with cooperative scheduling in scope of a single process.
The vast majority of them were already wrong. They only got more wrong.
You may just be used to knowing what code is "synchronous" and what isn't because it's been shoved into your face and you've adapted your thought process to it. In practice, "everything important is doing something 'asynchronously'" turns out to be the vast majority of what you need, and the vast majority of your mental energy you are dedicated to splitting the world in two is a waste. For the little bit that remains, by all means use something specialized, but it's just not something that everyone, everywhere, needs to be doing all the time, any more than everyone everywhere should be manually allocating registers, or any more than programs need to have line numbers because otherwise how can they work? (One of my favorites because I remember having that conception myself.)
Can you elaborate?
I don't want to be able to call fallible function from an infallible one trivially, I want the compiler to force me to specify what exactly I'm going to do with an error if it happens. Likewise for async-from-sync: there are many ways I could call these: I can either create a single threaded executor and use it to complete the future to completion, or maybe I want to create a multithreaded executor, or maybe I expect the future complete in a single poll and never suspend and I don't even need a scheduler.
I do love Rust and found a number of very valid uses for it but its async story leaves a lot to be desired. I don't enjoy writing it though I do enjoy the results.
Why do you think you don't need to know of it? I want to know if the function I'm calling is going to make a network request. Just because I can have a programming language that hides that distinction from me doesn't mean I want that.
Ideally I want to have the fundamental behavior of any function I call encoded in the function signature. So if it's async, I know it's going to reach out to some external system for some period of time.
That has nothing to do with function coloring.
> Ideally I want to have the fundamental behavior of any function I call encoded in the function signature.
There is no distinction of async functions if you don't have function coloring that you can encode in type signatures.
Sure, in the same way that types have nothing to do with enforcing logical correctness of software.
> There is no distinction of async functions if you don't have function coloring that you can encode in type signatures.
What are you trying to say with this statement?
1. inability to read an async result from a sync function, which is a legitimately major architectural limitation.
2. author's opinion how function syntax should look like (fully implicit, hiding how the functions are run).
And from this there is the endless confusion and drama.
The problem 1 is mostly limited to JS. Languages that have threads can "change colour" of their functions at will, so they don't suffer from the dramatic problem described in the article.
But people see languages don't fit the opinion 2, of having magic implicit syntax, and treat it as an equally big deal the dead-end problem 1. But two syntaxes are somewhere between minor inconvenience to actual feature. In systems programming it's very important which type of locks you use, so you really need to know what runs async.
I’m hesitant towards not distinguishing different things anymore and let the underlying system “figure it out”. I’m sure this could work as long as you’re on the happy path, but that’s not the only path there is.