That is a depressingly large amount of text to write about futures without referencing or even seemingly being aware of Notes on structured concurrency, or: Go statement considered harmful [1] or the Python library Trio that resulted from that. Trio doesn't have any future or task objects at all. To be honest, it is occasionally annoying (mainly for fetching the return value of a background task) but far far less often than you'd expect. For composing operations, you are generally much better off using using Trio nurseries (or task groups as asyncio calls them – having copied the idea from Trio!) and channels for communication between them if needed.
Sorry my blog post depressed you, but I wonder if it did so nearly as much as your arrogant & rude comment did me.
I am deeply aware of njs’s notes on structured concurrency & have been working on a post engaging with it. This post was not about structured concurrency & so it wasn’t relevant here.
Many of us loved the article but didn't have anything worthwhile to comment, so maybe one consolation is that the one rude comment is just the one that gets posted but not representative of people's overall reception to the article. You can't please everyone, as they say
Thanks I shouldn’t get so angry but I’ve spent most of the past year reading essays & interviews from the 70s to try to get my head around how to respond that specific blog post (which I think makes both good and bad points). Was very offended by the suggestion I wasn’t aware of it.
> even seemingly being aware of Notes on structured concurrency, or: Go statement considered harmful [1]
> The popular concurrency primitives – go statements, thread spawning functions, callbacks, futures, promises, ... they're all variants on goto, in theory and in practice. [...] Therefore, like goto, they have no place in a modern high-level language.
That's a really long and abrasive article ("Everyone else's stuff is GOTO, but mine isn't") to read just to get to "Wait for your return values before moving on".
No mention of any kind of synchronisation primitives, liveness guarantees, determinism, channels, transactions...
> His claim is that structured concurrency is the only way to make Ctrl+C handling work in a way how users naively expect it to work.
> And it’s easy to see why: If the exceptions are propagated up the call tree, freely crossing the thread boundaries, the KeyboardInterrupt exception is eventually going to reach the top of the call three (main function) and exit cleanly.
> However, there’s a hidden race condition involved: Imagine two threads in a scope, one gets KeyboardInterrupt and the other one raises an unrelated exception. If the latter exception bubbles up to the parent thread faster it will cancel the thread that’s processing KeyboardInterrupt and Ctrl+C would get lost.
> However, there’s a hidden race condition involved: Imagine two threads in a scope, one gets KeyboardInterrupt and the other one raises an unrelated exception. If the latter exception bubbles up to the parent thread faster it will cancel the thread that’s processing KeyboardInterrupt and Ctrl+C would get lost.
I don't think there's a race condition in what you've described. Trio and that article is about single threaded async applications, but it's true that it could apply to threads in principle. But, in that case, I would expect cancellation to only be propagated at specific points, much like Trio's checkpoints [1]. You can see exactly that concept applied to threads in C#'s CancellationToken object [2]. You can only cancel a C# Task (which can be running in another thread or asynchronously in this thread) by using one of those, and the cancellation only takes effect when the Task checks it (e.g. by passing the token to a suitable async routine, or by explicitly calling CancellationToken.ThrowIfCancellationRequested()).
So, in the example you gave, there's two possibilities: (1) A race happens because an exception is thrown right after a checkpoint (i.e., cancellation check) and the cancel exception gets thrown at that checkpoint - but that's not really a race (except in your application logic) because that's what you asked for. Or, (2), you get to the point of throwing an exception in one thread after cancellation has been requested from another thread (but before you got a chance to check it) - but in that case the exception will propagate to the top of the thread's call stack (if not otherwise caught) and out of the thread into your nursery-like concept. So you'll end up with a multi-exception containing the KeyboardInterrupt and the other thread's exception.
In the second case, there's a "race" in that you requested cancellation of a task but it finished before it had a chance to be cancelled. But that's always a risk, even with single-threaded async applications. I think it's just fundamental to the concept of cooperative cancellation.
(Aside: Personally, when using Trio in any application, I run it in a secondary thread, and I save the main thread for just waiting for KeyboardInterrupt (or for the secondary thread to finish naturally). That way, I can propagate the cancellation to the right bit of my application, which allows shutdown in an orderly fashion. I view Trio's default KeyboardInterrupt handling as just a decent attempt for those that aren't writing a serious enough application to need really reliable handling.)
func yield[G AnyGenericEvenInterface](promise G) G {...}
I'd have preferred the language expand yield() to take an argument which yields until the ((promise)d) variable is resolved to a value. This makes it more clear that the work is being passed off to runtime and will wait for that value.
I'm still somewhat confused that promise constructions don't have any well defined start time https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... They could start instantly or defer entirely until waited on by any of the resolution methods. It seems more complex than light threads / goroutines synchronized by communications queues and other first class language objects.
When I read this, I have to think of a take I heard a few times, namely that "Rust would be better off if the language had opted for a proper effect system from the get go, instead of grafting specialized forms of it onto the language now."
Does anyone have some thoughts on this? I just lack the knowledge to really evaluate that sort of take. There's a lot of discussions on whether async Rust is "good" or "bad", or the right abstraction or not, but it's pretty clear that (at this point) we're more less stuck with it (for better or worse), but that's not what I am asking about.
What I am asking is whether a proper effect system would have genuinely, and legitimately improved or significantly simplified the whole state of Rust async or not, and how. A lot of the async Rust problems sound like "hard" problems, and I'm a bit skeptical that a feature like that would really reduce the "heavy lifting" you have to do.
> Rust would be better off if the language had opted for a proper effect system from the get go
It does sound very appealing, but I'm not sure there's enough prior art to steal from, even now, but especially not when Rust was hatching. It would probably be too big a gamble. I'm checking out Koka in that space, but it's early days.
I've had to build something like an explicit futures system for a common case that doesn't look like an async pseudo-thread. There are a large number of requests for various assets. When an asset arrives from the network, all the objects waiting for it need to be informed so they can update. Individual objects may have multiple updates pending. Updates need not arrive in order. It's a many to many relationship, not a 1-1 relationship.
This is not uncommon in big-world games. I hit it in my metaverse client. It's a standard Windows feature. Windows file systems have "FindFirstChangeNotificationA", which lets a process monitor a collection of files for modifications. That has scaling issues for large numbers of files, but it's the same problem.
Similar ideas exist in the database world. You'd like to be able to ask "tell me when the results of this SELECT change". That's been played with as a concept, but didn't go mainstream.
In this area you can turn a manageable special case into a really hard generic problem. This is useful if you need a thesis topic, less useful if you need a working system.
I've worked with similar systems in gamedev. To be exact, we downloaded 2d images and then managed image atlases with them — so we needed to update renderers not only when image they required was downloaded and included in an atlas, but also when an atlas with their image was rebuilt.
To be honest, the system that we designed wasn't that dissimilar from promises. It was a dynamic collection of callbacks (using built-in C# events), not unlike then/catch delegates. I've worked on it about 10-12 years ago, exactly when JS world had discussions on what flavour of promises was the best, and it was one of the main sources of inspiration.
* Asset might be in memory in a cache. If so, satisfy request from cache.
* If asset is not in cache, queue up request, possibly on a priority queue that gets re-prioritized. Merge requests for same asset.
* When request reaches front of queue, see if the things that wanted it still want it. Remove dead requests. Discard entire request if all requestors no longer need it.
* Lock against two asset fetches for same asset being performed simultaneously.
* Fetch and preprocess asset
* Check again that asset is still wanted.
* Deliver asset
* Unlock against two asset fetches for same asset.
This is almost generic, except that once level of detail becomes involved, it gets less generic.
What I like about this post is that it provides a good attack against the function coloring philosophy, which always rubbed me the wrong way. Function coloring essentially implies that there's no good reason for async and sync to be different, and it's being used as an argument that we should try to sweep the distinction between the two under the rug as much as possible, and that's just not a good idea.
I also like the point that maybe(async) isn't terribly compelling. Recently, I've been looking at building an async parser, and the sync version of the interface is pretty simple: fn parse(r: &mut dyn Read) -> Result<ParsedObj> [1]. In theory, I could slap an async on it, change Read to AsyncRead, and now I have an async version. Except it's not usefully async: the consumer of the data can't do anything until everything has been fully read in. So what you want in the asynchronous version of the API is something that looks like a stream of parse events. But that API is way too much faff for anyone trying to use the API synchronously. Or is it? If you have a high-level method that distilled the stream of output events into a simple, easy-to-use answer, then you end up with something that looks maybe(async)-able [2].
Where I think the author gets it wrong, though, is in asserting that it's statefulness that causes maybe(async) to break down. State per se isn't the problem: our underlying parser above is statefully turning a stream of input data into a stream of parse events, and this could sometimes work well with maybe(async). The problem arises when you're multiplexing many streams at once. In synchronous code, there's just no way to do that kind of multiplexing (as pointed out earlier in the blog post), while it's a pretty key design feature of async code. The trouble really comes in when you have a system whose outer interface is a 1:1 stream interface, but which internally needs to go through a 1:N piece and an N:1 piece. It looks like it could be maybe(async)-able from the outside, but its implementation is hopelessly not maybe(async)-able.
[1] Honestly, a lot of parsers go a step further and just read in the input buffer as a &[u8] and rely on zero-copy, but that interface turns out to be even less useful in an async context.
[2] But the faff remains if you can't resort to one of the pre-canned methods. How much of an issue that ends up being is left to the reader to decide.
> If you were a language designer of some renown, you might convince a large and wealthy technology company to fund your work on a new language which isn’t so beholden to C runtime, especially if you had a sterling reputation as a systems engineer with a deep knowledge of C and UNIX and could leverage that (and the reputation of the company) to get rapid adoption of your language. Having achieved such an influential position, you might introduce a new paradigm, like stackful coroutines or effect handlers, liberating programmers from the false choice between threads and futures. If Liebniz is right that we live in the best of all possible worlds, surely this is what you would do with that once in a generation opportunity.
This seems... a bit specific? Was it a reference to a specific new language or was that more of a poignant wish?
Eh bien! mon cher Pangloss, lui dit Candide, quand vous avez été pendu, disséqué, roué de coups, et que vous avez ramé aux galères, avez-vous toujours pensé que tout allait le mieux du monde? Je suis toujours de mon premier sentiment, répondit Pangloss; car enfin je suis philosophe; il ne me convient pas de me dédire, Leibnitz ne pouvant pas avoir tort, et l’harmonie préétablie étant d’ailleurs la plus belle chose du monde, aussi bien que le plein et la matière subtile.
"Well, my dear Pangloss," said Candide to him, "when You were hanged, dissected, whipped, and tugging at the oar, did you continue to think that everything in this world happens for the best?"
"I have always abided by my first opinion," answered Pangloss; "for, after all, I am a philosopher, and it would not become me to retract my sentiments; especially as Leibnitz could not be in the wrong: and that preestablished harmony is the finest thing in the world, as well as a plenum and the materia subtilis."
Thank you. It’s been a while since I read Candide and while I say “All is for the best in this best of all possible worlds” somewhat regularly, that detail has since fled my mind.
> Rust chose this approach to get zero-cost FFI to the enormous amounts of existing C and C++ code written using that model, and because the C runtime is the shared minimum of all mainstream platforms. But this runtime model is incompatible with stackful coroutines, so Rust needed to introduce a stackless coroutine mechanism instead.
Can anyone (withoutboats?) elaborate on this? If “stackful” coroutines means functions with an ordinary-ish stack that can yield, then those stacks are just memory, and you can take references to them. Heck, you can do this, slowly and painfully, in C, with setcontext. And you can mostly do FFI with a degree of care about stack sizes. You can even yield across FFI boundaries. In some sense, the coroutine objects implicitly created by Rust and C++ are really just stacks minus the hardware stack pointer part.
Also, I found this amusing as a kernel programmer:
> the implementations of each of these things are completely different. The code that runs when you spawn an async task is nothing like spawning a thread, and the definition and implementation (for example) of an async lock is very different from a blocking lock: usually they will use an atomics-based lock under the hood with the addition of a queue of tasks that are waiting for the lock. Instead of blocking the thread, they put this task into that queue and yield; when the lock is freed, they wake the first task in the queue to allow it to take the lock again.
When blocking a thread, Linux (and most conventional operating systems) put the thread into a queue, mark the thread as non-runnable, and yield, and when the thing they’re waiting for is ready, they mark one or more queued threads runnable so the executor, I mean scheduler, will run them so they can try again :)
Most OSes do this a lot more slowly than can be done in userspace.
> Can anyone (withoutboats?) elaborate on this? If “stackful” coroutines means functions with an ordinary-ish stack that can yield, then those stacks are just memory, and you can take references to them.
See http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p136... for an in-depth analysis. The short version is that stackful fibers turn out to be a terrible choice. The author (Gor Nishanov, for the C++ ISO committee) explicitly says that they should not be used - this is even in spite of the fact that Windows API explicitly supports fibers as part of the programming interface, at a far deeper level than just POSIX setcontext.
Gor Nishanov's analysis is way too narrow to be used as general language design advice. Windows API fibers are also way too specific of an implementation to generalize.
Call stacks are just a data structure like any other. That they happen to be managed by the compiler to store things like local variables doesn't change this. Consider that there are algorithmic ways to convert from function calls to this data structure (defunctionalization) and back (refunctionaliztion), and the correspondence between recursive programs and iterative programs using an explicit stack.
The question here is, for any given stack, what requirements does it place on the memory layout and allocation for that stack? Gor's analysis covers a few of these, but not everyone has the same requirements and not every implementation style makes the same tradeoffs.
And you can mix and match these styles within a single programming language! Rust async functions and C++ coroutines both still store much of their local state on the native C stack- they only use their state machines to store local state (including the state machines of their callees) that lives across suspension points. This is what enables Rust to support APIs like spawn_blocking or block_on- you can run different parts of a program using different styles of stacks.
> Call stacks are just a data structure like any other.
Call stacks are a part of the system API/ABI; your fancy ersatz stacks not so much. This is kind of a key consideration in the above analysis. The whole point of having Windows API support for fibers was to try and obviate that basic pain point, but ultimately this has been shown not to be very successful.
> Rust async functions and C++ coroutines ... only use their state machines to store local state ... that lives across suspension points.
Yes and this is briefly mentioned in that analysis as a key advantage of stackless coroutines compared to fibers. If you might call a function that requires 500k of stack, a fiber has to provide for that amount of stack space to be available; a stackless coroutine doesn't need to store the 500k of data, provided that it doesn't live across await points.
Fibers can make that exact same optimization by using a separate stack (as a stackless coroutine uses a separate state machine) alongside the C stack. This can be done either through stack switching, or by treating the fiber stack more like an ordinary object without a special reserved ABI register.
The reason Windows fibers failed is not because they were stackful, but because they tried to use only one stack at a time.
This would seem to be comparable to segmented stacks - which turn out to be fiddly enough that Golang got rid of them. And they're said to be even less feasible in languages that don't use GC and can't adjust pointers to account for moved data.
Segmented vs contiguous stacks is a completely orthogonal design choice here. The key is merely that this stack is separate from the C stack but accessible simultaneously.
As long as you have that, it can be segmented, or it can be contiguous and movable for growth if you have a GC and/or forbid pointers to locals, or it can be perfectly sized if you do a whole-program analysis on stack usage, or it can be large and non-moving like the C stack.
The bigger problem is growable stacks. It’s true you can suspend a stack and then continue as long as your implementation doesn’t ever move it. I didn’t really dwell on this because I wrote more about it in “Why async Rust?”
And you’re right that the mutex in the kernel is implemented with a similar algorithm - I just meant that the code in the library you’re using is very different, because it is a relatively thin wrapper around the relevant syscalls whereas the async one is implemented in userspace.
Too often people think the problem between sync and async is a red/blue coloring within the programming language. The problem really is that every OS already has red/blue syscalls.
Everything else about “why can’t I mix these functions” is a direct result of that broken consistency exposed by the OS. As such I’m not sure there is a zero-cost abstraction (stackful coroutines is close) that any programming language can build to improve the situation.
Meanwhile, the trend is clear - every OS has adopted (and is adopting more) non-blocking syscalls because they are direly needed. The only benefit blocking syscalls offer is sugar to improve syscall ergonomics.
I think if more people talked about it this way it would become clear that adding a “blocking” tag to the syscalls and bubble that tag up the call stack is the right next step to depreciating those legacy OS APIs. I don’t mean to say we should accept poor ergonomics but adding the blocking tag is a great first step to 1. Reminding people of the problem. 2. Identifying areas where research is needed to improve ergonomics and replace the existing “blocking” tag with minimal downsides.
Really good critique of the current state of handling, but my opinion is still that the language should not itself makes any design decision on async. I believe everything should just keep being a set of instructions that are executed accordingly to the machine, and that's it. It helps ensuring the language semantics are still relevant on newer execution machines.
Indeed, async v. sync should be a matter of the application and the application only.
This is like arguing for goto over structured concurrency. Async/await is a structured control flow primitive that lets you compose functions in more flexible ways. It will stay relevant as long as we still use things like functions, branching, and looping.
The post makes a good point, albeit in a long-winded way.
My short summary would be in languages that result in function-coloring due to async/await, using Future/Promise directly avoids the coloring and the ecosystem is better off.
Places where async/await are ok are JS (because both the browser and node/deno) are for the most part in an async context, and Go since everything's a goroutine hence all one color.
Languages such as Java have worked well with Future/CompletionStage and doesn't really need the convenience of writing async/await which translates the rest of the program after await into a hidden 'rest of program' callback. That async/await syntactic convenience is not worth the complexities added to libraries. Rust including async/await was a poor 'cargo cult' choice. I would welcome a Rust fork/ecosystem without it.
That’s absolutely not a summary of my blog post, since I completely disagree with it. I think async/await is great and avoiding it for some reason would do nothing to mitigate the function coloring problem.
If you're a language/platform like Swift or Rust (see also Erlang or OCaml) you're gonna end up with some kind of userspace task scheduler because OSes don't give you enough control over threads. If you're a systems language you're gonna use stackless coroutines because speed/memory and C interop. In Rust, you use their task scheduler via async/await, but sadly the ergonomics aren't wonderful and there's now weirdo problems like what reactor/engine/whatever they standardize on etc. The interesting and good stuff will be built on top of async/await, probably shouldn't have touted it as the interface.
My hypothesis here is that something like concurrency/parallelism exists on two levels. You've gotta get the underlying implementation right, and you have to expose a mental model and affordances for using it. The problem is that languages that are good at the first thing are usually real bad at the second thing. More broadly I think this is a trilemma--pick any 2 for your language:
- small
- can implement performant concurrency/parallelism
- can express a high-level mental model of concurrency/parallelism
No one should be surprised that Rust will throw "small" overboard here. I think the only question is how many models will they support? Will they do CSP AND actors (they're different!)? Will there be some kind of OpenMP type thing? Will there be new keywords or--shudder--new SYNTAX?
---
I feel like I have to say Go's solution to this is bold and elegant. Giving up straightforward C interop was pretty gutsy, and the affordances for working with its task scheduling system are familiar and intuitive. I want to stress this (because TFA goes way out of its way to be a little shitty w/ Go): this worked absolutely great. Tons of good software is written in Go. It is super effective. It spent its innovation tokens on experience and tooling and that was really successful (i.e. type systems aren't the only way to fight data races). There were exactly zero debates about stuff like await syntax or what engine to use/standardize. In the time it took Rust to ship async/await, probably 7 billion actually useful Go programs were shipped. Just because someone else's values aren't yours doesn't mean they're the wrong values. I think it's cool that people like Boats are being really thoughtful about how to do this and sharing their thoughts and processes out in the open. I just think there's room in this crazy world for at least two points of view, and that we could do with a little less sneering.
> Will they do CSP AND actors (they're different!)? Will there be some kind of OpenMP type thing?
Rust builds support for all of these models based on it's basic underlying "shared ^ mutable" and "Send/Sync" safety guarantees. It should be quite clear based on this whether you're sharing access to some read-only data, staying within a single-threaded context with mutability, or relying on some sort of synchronization primitive (be it atomic data, locks, r/w mutexes etc). The fact that we understand some uses of these facilities as involving patterns such as "CSP" or "Actors" is quite beside the point.
I don't super understand your point. I understood TFA to making the argument that dealing with the low-level business of stuff like mutexes and async/await isn't what most people want to do:
> We should aspire not to simplify the system by hiding the differences between futures and threads, but instead to find the right set of APIs and language features that build on the affordances of futures to make more kinds of engineering achievable than before.
The stuff you listed, along with async/await, makes this possible. Isn't that what TFA is saying?
It seems to me that there are two huge (cosmetic? branding?) problems with async/await
1. The words are completely awful. There's nothing "not synchronous" about an async function. Maybe at best "not synchronized".
2. The idea of pretending async functions are still "functions" is kinda terrible.
This is really a huge problem in say python where the architecture (iirc) is via iterator/yield, so your function "actually" has multiple outputs.
Maybe a radical idea in PL design would be to actually make "async functions" be some sort of other thing in the same sense that, for example, pascal had "procs" and "functions"
You could easily solve the coloring problem by creating some sort of builtin that does an immediate, blocking execution of the function by compiling out all the suspend sites, or keeps them "in place" in the case where you want to keep things as if they are futures.
Not necessarily. That's (mostly) true in e.g. JavaScript, but is not true in Rust.
My novice understanding of the Rust model is that an async function is compiled down to a state machine and a set of functions which execute parts of the async function's code and then manipulate the state machine. Those manipulations have two broad categories of side effects: they can wake up/be observed by pollers, and can change the state of later parts of the task corresponding to an invocation of the original async fn (e.g. by setting variables for subsequent functions in the set--code on the other side of an internal await point that uses the result of the await).
In other words, async functions in some languages are more than just sugar for "delegate to event loop and return promise resolved by delegate completion".
I don’t really understand what you mean about side effects but async functions do indeed just return futures just like JavaScript async functions return promises - rust futures just are compiled into state machines in a way JavaScript promises are not.
What about higher order library functions, since calling a green function from a red context is bad and we have no way to guarantee that a function is blue and not green, does this mean the function needs to be unsafe? It would be nice if the conversion from red to green would be done implicitly by the language where there is no ambient executor available, but I'm not sure what would be the implications for intra-task concurrency.
Part of the point of effect systems is precisely to allow HOF's to deal with these "color" differences in a consistent way. Similar to how in Rust, an error return is modeled as just an instance of some Option<> or Result<> type instead of some bespoke 'exception', so all HOF's can seamlessly manage the 'exceptional' case.
This might be a silly question, but what would be the downside of a single threaded language similar to javascript, but where every function is just 'async capable' and can always be non-blocking, with no special syntax like async/await?
So every time you call foo(); you anticipate that it might resolve immediately, or it might take some time, but it won't block any other function (unless it actually does some cpu bound operation)
And any time foo(); does something asynchronous, all of its callers, and its callers callers, become implicitly asynchronous too.
Of course you would have to have some primitive for when you actually want to do several things concurrently within the scope of a single function rather than blocking that function, but that doesn't sound too bad and in JavaScript you effectively need to use Promise.all most of the time anyway.
I'm sure there's some major downside I'm missing -- but what is it?
Is stackfulness required for what the grandparent describes? It seems possible to do this stacklessly: all functions implicitly compile down to a state machine, but futures are never visible.
You need stackful coroutines or continuations (which become really just a kind of growable stack) if you want recursive functions. This is a limitation of Rust’s design.
Generally speaking async code in a language that isn't ground up designed around that is far slower, because the language ends up using some relatively expensive mechanism like mutexs. In most languages making 90% of your code 90% slower but "everything async" isn't really a worthwhile general trade off.
Even is your language run time is fully async, it's really hard to do anything useful without touching C/C++/the kernel/system libraries, and as soon as you do that all those guarantees go out the window, because you cannot rewrite the world in $newlang.
You can do that, Java did recently. You need full control of all the blocking primitives so they can yield the thread.
Where this breaks is with FFI: if you cannot intercept blocking calls in foreign functions, you block useful threads or even deadlock. This is what the quote in the article is about:
> The cost for the native compatibility with the C/system runtime is the “function coloring” problem.
I just want a language where await is the default behavior and I have to specify when I want to capture a future as a value and do something with it later. Most of the time I want an emulation of a synchronous thread of execution even if it's actually async. It's crazy to have to constantly specify the common case and to have the possibility of surprising behavior at best if I accidentally forget to await something somewhere.
"T" and "impl Future<Output = T>" are very different types so unless you're doing very silly things I don't think you can be "surprised" in Rust. The fact that await points are explicitly marked and visible in the code is a feature, not a bug. It's similar to the case for "?" vs. hidden exceptions.
i think that if someone is really interested in this topic, they'd do well to read mark s. miller's doctoral dissertation on it, 'robust composition': http://www.erights.org/talks/thesis/
this is relevant because miller explains why he designed promises the way he did, although since he wrote the dissertation in 02006, he couldn't yet explain how he adapted them for incorporation into the javascript standard. following his dissertation, he was on the ecmascript committee for a number of years, which is how we got e-style promises in javascript, although obviously that also involved the committee as a whole being convinced that they were a good idea. (much of the dissertation is actually about object-capability security, but chapters 13–19 are about promises.)
the great benefit of promises is that you eliminate data races, where one procedure is correct only if a particular piece of mutable data remains unchanged, while another concurrently runnable procedure changes it, so the interleaving of their accesses can result in a failure. consequently, you don't need locks or queues to communicate between different concurrent procedures; you can just use a shared data structure with no locking. this is because promises are an alternative to threads. if you're familiar with async rust, you will note three glaring problems with this:
1. rust eliminates data races through the type system, so the great benefit of promises doesn't exist in this context
2. tokio runs multiple tasks concurrently, so if you do have data races, promises won't save you
3. so you do in fact use channels and locks in async rust
so i am puzzled about why you would ever want to use promises in rust, and i guess this post is an attempt to answer that question. it explains why people have this bizarre misconception that promises are just an efficiency hack, because in rust that's all they are, and those people have never used promises in javascript or e or the other children of e such as spritely goblins.
this bears repeating. before reading this post i didn't realize this, but rust async/await is just an efficiency hack. even the section of this post entitled 'i don't want fast threads, i want futures' never brings up, as far as i can tell, any correctness or flexibility advantages for async rust, only efficiency advantages ('to multiplex arbitrarily many perfectly-sized tasks on a single thread', 'much better performance characteristics' etc.) there's brief lip service to concision ('concurrency-related boilerplate about spawning threads', which i don't think actually exists) before returning to the main theme of performance. then it proposes some truly astoundingly terrible ideas about automatically introducing race conditions if you await with a lock held, in the name of being 'even more optimal', at which point we're well into 'mongodb is web scale. if /dev/null is web scale i will use it' territory
a terminological quibble: i wish people wouldn't refer to promises as 'futures'; a future is, or anyway originally was, a different approach to the concurrency problem, one where reading from a perfectly normal variable would block your thread until that variable's value had been computed (allowing other threads to run). this approach does not have the advantages of promises. promises are what async rust implemented, just under the name of 'futures'. really, though, confusing terminology seems to be the least of the problems with rust's implementation of the paradigm
> 1. rust eliminates data races through the type system, so the great benefit of promises doesn't exist in this context
I don't know about this one. Programmers who use async Rust often talk about how they appreciate that `await` is explicit because it lets them coordinate state using cooperative scheduling. This is maybe a bit higher-level than just data races, but it is about avoiding certain kinds of race conditions.
> it explains why people have this bizarre misconception that promises are just an efficiency hack, because in rust that's all they are
I'm not sure about this either. The post argues that they are more than an efficiency hack in Rust, too- that intra-task concurrency is a useful control flow structure and promises are what enable it. That intra-task concurrency happens to share a thread is a performance benefit but that's incidental to its flexibility benefits.
> promises are what async rust implemented, just under the name of 'futures'.
Well, it's not like Rust's promises are exactly like JavaScript's promises either. JavaScript promises are continuation/callback based; Rust's are poll-based. In a sense they're like your earlier definition of "futures," except that the future API lets its caller decide how to block when the result is not yet available.
i appreciate the feedback, and you could be right; do you have an example of how you can coordinate state using await?
i don't agree with your implication that data races are a low-level thing. they can be, but the canonical example of a data race is a lost write to a bank account, which is not just layer 7 (the application layer) or layer 8 (the user) but layer 9 (the economic production system that employs the user)
> JavaScript promises are continuation/callback based; Rust's are poll-based
this is obviously important for the implementation of a so-called async runtime such as tokio, but as i see it, from the point of view of most of the user code, this is an implementation detail. i was talking about the semantics promises provide to things like an http client, not how those semantics are implemented
> the future API lets its caller decide how to block
it's not clear whether in this phrase you are referring to the real future api or to what rust confusingly calls 'futures'. in case you mean the former, i want to clarify that, in the futures interface defined in 'multilisp: a language for concurrent symbolic computation', (halstead, 01985, https://dl.acm.org/doi/pdf/10.1145/4472.4478), you cannot distinguish a future from the value that it eventually produces, and therefore the caller cannot decide how to block; it doesn't even know it's blocking. as halstead explains:
> An operation (such as addition) that needs to know the value of an
undetermined future will be suspended until the future becomes determined, but
many operations, such as assignment and parameter passing, do not need to
know anything about the values of their operands and may be performed quite
comfortably on undetermined futures. The use of futures often exposes surprisingly large amounts of parallelism [now conventionally called 'concurrency'] in a program, as illustrated by a Quicksort program given in Figure 1. (...)
> These “futures” greatly resemble the “eventual values” of Hibbard’s Algol 68 [39]. The principal difference is that eventual values are declared as a separate data type, distinct from the type of the value they take on, whereas futures cannot be distinguished from the type of the value they take on. This difference reflects the contrast between Algol’s philosophy of type checking at compile time and the Lisp philosophy of run-time tagging of data.
i would claim that, since halstead's paper has been cited 1658 times in google scholar, including 25 times in the last year, his definition of 'future' is very much in current use, and so it is sort of an act of intellectual vandalism to promote a conflicting but confusingly similar definition
i'd still be delighted to see examples of how you can coordinate state changes with 'await'. git repositories containing purported examples would be fine, i'm not suggesting you write an entire async rust program in an hn comment
from my point of view, at this point, rust async seems to be a terrible design blunder. rust's design for massive concurrency is statically proving race-freeness, and that's what makes rust such a promising development. async in rust seems like the kind of development that could destroy the community and render the language worthless, while simultaneously torpedoing public understanding of halstead's futures model (though i don't think it's very promising) and javascript's async approach to concurrency (which is), ruining their chances to succeed as well. so i'm highly motivated to investigate any evidence that i'm wrong
> rust async seems to be a terrible design blunder [...] that could destroy the community and render the language worthless
What an extreme take!
Async Rust as-it-is makes perfect sense for Rust's primary audience: folks who desire control/performance enough that they would otherwise be using C/C++. To call async an "efficiency hack" is to underestimate how important that efficiency is to Rust's existence. If async didn't work the way it does, it wouldn't have gained traction within the Rust community (folks would have just ignored it and kept writing state machines by hand) and it wouldn't have been able to act as a draw for C/C++ developers. It's a killer feature, and has enabled the language to thrive.
i'm not underestimating efficiency; i'm well aware that in many situations it's more important than correctness. but i think it's important to distinguish between decisions made for the sake of efficiency and decisions that tend to make your code more modular, flexible, comprehensible, verifiable, or maintainable. promises originated as a way to make highly concurrent code more modular, flexible, comprehensible, verifiable, and maintainable. halstead's futures did, too. so did rust itself
async in rust is the opposite, and to me it looks like such an expensive way to get that efficiency that it strongly undermines the reasons for adopting rust
> even the section of this post entitled 'i don't want fast threads, i want futures' never brings up, as far as i can tell, any correctness or flexibility advantages for async rust, only efficiency advantages
I don't think that's charitable. While there's not, like, a concise bulleted list of async's advantages other than performance, many of those advantages are discussed throughout the piece:
- Cancellation (and the corollary, the ability to externally control, suspend, or time out arbitrary sets of computations at yield points).
- Ergonomic (rather than just performant) heterogenous multiplex-waits. Doing that in non-future-oriented systems isn't just potentially slow, it's fiddly and a breeding ground for bugs in many languages, including Rust.
- The ability to outsource the geometry of concurrency and parallelism to an async runtime rather than spending brain and editor cycles deciding things like e.g. how many handler vs. acceptor threads you're going to manage, and whether executors will launch/stop lazily or eagerly.
Those aren't necessarily unique to Rust, but they are definitely discussed and are definitely unique benefits of an async/promise-based concurrency system.
Also, minor nit:
> tokio runs multiple tasks concurrently
I think you mean "in parallel" here; the whole meta-system is concurrent, but data races re-emerge in Tokio (or rather, impose sizedness/pinnedness restrictions on data in async contexts in Tokio) where they aren't in JS because Tokio can run in true threaded parallel.
it's possible i'm just wrong about some of these things because of lack of experience with async rust, but i'm not convinced by any of your examples
> - Cancellation (and the corollary, the ability to externally control, suspend, or time out arbitrary sets of computations at yield points).
i'm not convinced by this one; i'd need to see more details, but my intuition is that it's just an efficiency hack, and i'll explain why
in a shared-nothing system like erlang (or pre-threading unix) you really can externally control, suspend, or time out arbitrary sets of computations. if you have to wait for a yield point, you only sort of can; a task stuck in an infinite loop that therefore never yields will not get a chance to be canceled. moreover, if a task is holding a lock when it yields, that could be either because its own correctness depends on some shared mutable data structure staying consistent, or because it currently has such a structure in a temporarily inconsistent state that it needs to restore to consistency
in the second case, how do you cancel it? you need to somehow restore the consistency, and nobody outside the task has the information required, and if it just releases the lock and exits instead of continuing, the next task that depends on the inconsistent structure will probably crash. so you can't cancel it at the next yield; you have to wait until the task yields without any locks held. and similar remarks apply for suspension, except that the potential problem is not crashing but a deadly embrace, a particularly pernicious kind of deadly embrace because it's hard to tell who was responsible for unsuspending the task
but let's suppose that being able to sometimes cancel, suspend, or time out a computation is a useful feature. we must ask, then, when is it useful? if a task is computing a result that is no longer needed (perhaps the user has already been shown a 'timed out' error, for example) what is the harm of allowing it to continue as long as it wants? it depends on the task, of course, but it had better not be crucial to correctness, for two reasons. one is that, as explained above, the cancelation might have to be delayed for an arbitrarily long time so that the task can suspend while not holding any locks. the other is slightly more subtle: if the task's correctness depends on it getting canceled early enough, then its correctness is dependent on the other task that kills it getting scheduled early enough, which is generally not something that can be guaranteed
consequently, i believe the only case where task cancelation can be useful in a shared-memory system like this is when it's just an efficiency hack and doesn't actually affect the system's semantics
but maybe there's something i don't understand about this (my ignorance of async rust is truly vast) so please let me know. git repository urls containing purported counterexamples would be especially welcome
> - Ergonomic (rather than just performant) heterogenous multiplex-waits. Doing that in non-future-oriented systems isn't just potentially slow, it's fiddly and a breeding ground for bugs in many languages, including Rust.
it's possible i'm not understanding what you mean; can you explain with code? it sounds like the kind of thing you could put into a library function in a system using multithreading; you can solve it without reconsidering your entire programming paradigm
> - The ability to outsource the geometry of concurrency and parallelism to an async runtime rather than spending brain and editor cycles deciding things like e.g. how many handler vs. acceptor threads you're going to manage, and whether executors will launch/stop lazily or eagerly.
this one is purely an efficiency hack; in a threaded system you can spawn a new handler thread for each request which terminates when the request is done. maintaining a thread pool is itself purely ...
I think you’re too quick to dismiss the utility of yield-point cancellations. You’re right that yield points are a coarse means of imposing suspension/cancellation. However, I think they’re often preferable to the alternatives. Erlang, for example, imposes significant overhead due to reduction counting and restrictions on copy-free data sharing in order to achieve (nearly: dirty NIFs are forever) complete arbitrary subroutine control from the outside along with progress guarantees.
From a certain perspective, the knowledge that suspension/cancellation can only occur at yield points shrinks (or at least makes somewhat more vislble) the risk surface of areas where arbitrary cancellation can leave the program in a corrupt state.
I do agree that more work is needed in this area to ensure automatic state reconciliation after cancellation (the mutex-held-during-await issue). Python makes a decent stab at this: its async/await concurrency implements cancellation as an exception raised at await points, allowing cleanup to occur before proceeding with cancel. Unfortunately, that comes with the consequence of allowing routines to choose to defer cancellations, or ignore them entirely. Perhaps that could be mitigated with an async/await system that makes cancellation a bidirectional exchange, changing observable behavior at the canceler when the cancellee does or doesn’t actually terminate? I think the Rust folks are moving in this direction with async-drop, though I personally do not like the idea of implicit custom destructors in the first place, and worry that allowing them to effect async cancellation control flow will hurt rather than help. That’s a separate topic, though.
All that said, I think that struggle is a somewhat inevitable outcome of a platform that supports shared memory between routines and C interop. If you’re willing to give up on those, then sure: you can preempt or cancel with much more flexible semantics. But I don’t think that’s innately better: first, C interop is critical for some platforms, as Boats mentions. Second, while you might be right in pooh-poohing some of my other points as “just efficiency”, adopting an Erlang memory sharing model is really expensive—impactfully so for a much wider variety of programs than the ones for which, say, stackful vs. stackless coroutines make a difference.
On to your next point: how important is yield-point cancellation, really? I think that
> the only case where task cancelation can be useful in a shared-memory system like this is when it's just an efficiency hack
isn’t correct.
Firstly, “efficiency” is kind of a moving target. Is it merely efficiency if we can, say, take advantage of the fact that memory used by cancelled coroutines is freed once they’re cancelled? Consider this code:
If cancellation of the first call to memory_hungry() did not trigger the drop of x (90% of system RAM), then we’d be unable to successfully call memory_hungry() a second time after the select!, since that would put us into OOM.
Like, sure, there are all sorts of ways to accidentally leak resources in the event of cancellation (we could mem::leak it, could have memory_hungry() spin and block the task, and so on). But this is one fairly happy path where a very important resource (memory) is correctly managed by the cancel-at-yield model.
Secondly, what about the cancellation of routines that keep something alive? For one example, consider routines that send heartbeats or liveness probes: cancelling those can be very meaningful to the program’s semantics. If your rejoinder is that that’s a bad idea because of the chance of a while-true loop blocking routine causing a spurious event loop hang, I’d respond that we’re either now discussing a whole different set of constraints (RTOS behavior...
Thanks for engaging constructively, and for the detailed responses. This whole area is fascinating to me, especially the history of how we think about concurrency from Milner ("Calculus of communicating systems") to Hoare (CSP) on down to the present day, and I really enjoy talking about it. If you'd like a less asynchronous (ha!) dialogue, feel free to reach out at my username but with two "b"s instead of one at gmail.
1. I updated the gists a bit for clarity, hopefully without accidentally moving any goalposts.
2. In addition to the tradeoffs mentioned in the sentence "If those aren't priorities for you...", I didn't really respond directly to your points about loop-blocking causing event loop hangs.
The possibility of loop-blocking causing the entire concurrency system (rather than just, say, one request's routines) to malfunction is a real drawback of async/await systems. Whether or not it's a deal breaker for your using such a system boils down to trust, I think. If your program is running a lot of background routines whose behavior you don't trust to have a low likelihood of loop-blocking, then async/await might be problematic for you. This is analogous to deciding whether to include a piece of functionality as a library linked into your main program, or to include it as a subprocess/IPC interface instead. If you don't trust that functionality not to e.g. corrupt memory, you may choose the latter.
I think Tokio specifically is better at mitigating those drawbacks of async/await concurrency models than many other systems, due to its abilities to a) provide fine-grained and accurate detection of loop-blockers, and b) to move task execution between threads to get around potentially stuck routines. Those are just mitigations, not solutions, but they're substantially better than what's offered by e.g. JavaScript.
I’m not familiar with E’s promises, but as far as I know, JavaScript promises have no equivalent of heterogeneous selects (select in Go, select! in Tokio, sync in ConcurrentML) that guarantee that when one channel/future/event is chosen, the others definitely do not make progress (I.e. a select on a read and a write means only one of them happens on each turn). That seems important.
Promises don’t prevent data races. They only don’t exist in JavaScript because the runtime is implicitly single threaded.
it's true, being implicitly single threaded is what prevents data races; promises are an alternative to shared-memory multithreading that does a better job of preserving that property than shared-memory multithreading does
if you want to select on multiple promises, it's relatively easy to do in javascript; you make a new promise for the disjunction of the multiple promises (e.g., using .withResolvers()), and attach a callback to each of the original promises that resolves the disjunction promise when it fires
this of course does not prevent the other promises from getting resolved, because that would happen in some other piece of code, perhaps in response to a network i/o event or a filesystem i/o event; knowing whether a particular turing-complete event handler is going to resolve a particular promise (so you could avoid running it) would require solving the halting problem
> this of course does not prevent the other promises from getting resolved, because that would happen in some other piece of code, perhaps in response to a network i/o event or a filesystem i/o event; knowing whether a particular turing-complete event handler is going to resolve a particular promise (so you could avoid running it) would require solving the halting problem
It does not if you control the implementation of the language primitives ;) I’m not smart enough to answer exactly how, but Andy Winford explains how CML does it using the notion of optimistic and pessimistic stages. https://wingolog.org/archives/2017/06/29/a-new-concurrent-ml
thanks, this is a really interesting article! but it seems to be about something different
fundamentally one of the major things promises are useful for is waiting on network packets which may never arrive. (in the small, sometimes the network in question is sata or usb, but that makes surprisingly little difference.) there's really nothing you can do to keep some other host from sending you a response packet to a request you've already sent it, and when your network interrupt fires, unless you've brought the network interface down entirely, your interrupt handler needs to do work to buffer the packet (if necessary) and inspect it to see where to route it to. if that turns out to be a promise, all of this is 'making progress on' a promise that may be canceled or whose fulfillment may have no effect. (you could, however, limit the resources you dedicate to such useless work: an important optimization in many contexts, but not one that needs language-level support)
so even controlling the implementation of the language primitives doesn't help
Your point is entirely valid. I had not thought of it. Makes sense for resource consumption.
I think selection is about what is visible to your program. I.e. that even if that packet arrives, if you choose another path in the select, then the recv() is guaranteed not to happen. You are right that a program doing this has to handle this case correctly, which I think is what the whole Rust future cancellation conversation is about.
(b) threads support some form of cancellation (with an ecosystem that expects this and is built around it),
then you don't need "intra-task concurrency" as a fundamental primitive. You can achieve the exact same semantics by just having a helper function that, given N closures representing work to perform, spawns a new thread to run each one, and waits for any/all of them to finish, while making sure to cancel all those threads if it itself is cancelled. (Or if you want to manually manage future objects for more control, you can do that too. But a future just represents work being done on a thread.)
Compare to Go. Go has cheap threads. Boats criticizes Go's "threads, locks and channels", but the fact that you have to interact with those primitives directly, rather than using a helper function as described, is really a limitation of Go's type system, not an inherent result of basing concurrency on cheap threads.
With one exception: Go threads do not support cancellation. Cancellation is extremely important! Without it, you have to manually pass down timeouts and other forms of cancellation into each function that performs I/O, and it becomes impossible to write that kind of generic helper function.
And of course, Rust has to fit into an existing thread ecosystem that does not meaningfully support cancellation, and does not make threads cheap.
So the tradeoff for Rust remains. Still, I don't see this anywhere near the way boats sees it. In my ideal world, if I didn't care about performance or compatibility, I wouldn't want "a language in which all functions are coroutines, meaning that all functions can yield." I don't think yielding needs to be visible to the user in any meaningful way; why does it need to be? It can be a hidden implementation detail, just like it is with green threads or OS threads. But I'd add on a concept of cancellation.
…In short, I think my ideal model is something like Trio, but I haven't had the opportunity to actually use Trio. I hope that changes.
(edit: and I'm looking forward to when boats does respond to the Trio blog post.)
Does synchronous Rust (with no runtime and no tokio) have a cross-platform way for a parent thread (eg. a GUI) to wait for messages either from user input or a child thread (eg. audio device changed), and/or a child thread to wait for messages either from IO (eg. audio buffers ready to fill) or other sources (eg. audio timeout or cancellation as you mentioned)? In C++ on Windows you'd use WaitForMultipleObjects, on Unix you'd use poll on hardware/event file descriptors, Crossbeam channels have a select macro but I'm not sure if it's as real-time as WaitForMultipleObjects (officially recommended for WASAPI) or poll (officially recommended for ALSA), and it probably doesn't work outside of channels?
Take a look at ConcurrentML and its implementations in Racket and Guile. Schemes have stackful coroutines (as a side effect of supporting delimited continuations), and CML builds green thread like semantics on top of that with a small set of elegant primitives, including cancellation.
As an aside, when folks mention algebraic effects in statically typed languages, delimited continuations and effects are equally powerful (possibly identical?). IMO effects seem to be about type safety and static typing, which does allow some neat optimizations.
This is interesting and makes sense. It's worth mentioning that the original use of structured concurrency (by Sustrik) was really about dealing with the problem of cancellation-friendly virtual threads: https://250bpm.com/blog:71/
> I don't think yielding needs to be visible to the user in any meaningful way; why does it need to be?
It's an interesting question. Users definitely want coroutines they can yield themselves - this is was iteration is all about, for example, and there are other similar patterns (the entry API is ultimately a kind of coroutine).
At the same time, the highest performance low level interfaces for asynchronous IO are ultimately just asynchronous iterators & asynchronous sinks over shared memory ring buffers, io-uring is the most famous example but there are others like AF_XDP.
I think you're right that a kind of cancellable virtual thread ultimately would have analogous affordances to a future. But a future is just a specific class of coroutine (that yields a unit type), so if you will have the coroutine mechanism for the rest of the system, why would you not extend it to include IO instead of adding a separate mechanism to erase the fact that IO is happening? What you also get from this is that if a function yields the bottom type you know it doesn't perform IO.
Terrific post. I think several of the more disagreeable commenters here are engaging with it piecemeal and projecting stronger opinions onto it than it actually has. I interpreted it as more of a review/analysis/set of musings by a qualified expert in an area than a bunch of decisive hot takes.
It definitely makes me wonder if the original sin of Rust and many of the other colored-function-based concurrency systems was limiting the enforcement of color-can-only-call-color to only async functions. Would a language that disallows calls back from async functions into blocking functions make sense? Probably not. But what if we split "blocking" into two categories: functions that provably blocked for constant time (or even constant-less-than-X time; simple systems like BPF can verify that already, so I'd assume it's possible to get more nuanced here with effort) versus functions that did not (including I/O, while-true-with-non-statically-verifiable-condition)? If we disallowed calls from async functions into functions of the latter category but not the former, would that be ergonomic/accessible for programmers?
(A minor quibble/criticism of an otherwise great post: I think the discussion of "blocking functions" should get a crisp definition up front, qualifying that "blocking" in this case means undesirably long or potentially indeterminate blocking of executor threads, so readers don't get confused and wonder whether the implication is that any synchronous function call is inherently--rather than just potentially--toxic to good async behavior).
Continuing to think about the idea of "async code can't call 'bad blocking' code": I suspect that one of the reasons this isn't widespread is that it really does require compiler/runtime validation that code down a call stack is good/bad. Erlang almost got there, but (just like Boats points out happened with Rust) the C FFI chimera stopped them in the home stretch and they settled for asking users to tag dirty vs. clean NIFs. But maybe compiler awareness of call path blocking risk is advanced enough today that such a system could be implemented? It definitely wouldn't be simple (...he said, as if he were anything but a rank amateur when it came to understanding compilers). It definitely would solve the mutex-held-across-await-point class of problems.
Two other points that stood out to me that I thought were worth mentioning:
1. The rejoinder to the blocking-reqwest-vs-ureq situation is well put. Honestly, Boats could have gone further here: I think that a large proportion of the complaints/reluctance around using synchronous APIs that wrap block_on or whatever are fundamentally aesthetic/emotional objections arising from people's bad vibes around including or separately instantiating an async runtime inside a library just to provide a blocking API. In practice, the compile overhead is rarely severe and nearly always paid extremely infrequently, and the runtime overhead in time/memory is negligible in release builds. Supporting the emotional resistance here is, I think, a lot of user ignorance around the practical differences between, say, how a current_thread Tokio entry point runs Rust futures and how a libuv loop or a boost::asio context or whatnot run futures in JS/C++/etc. While Rust and Tokio aren't magic (and while I would love more crates to be async-to-syncable with pollster alone), I think that folks often project assumptions about heaviness/performance from other systems onto parts of the Rust ecosystem where they don't hold as true.
2. The Golang callout is fair (and was hopefully cathartic to write), but in addition to being punchy I think there's a lesson to be learned there, albeit a bitter one: Go is widely considered to have wildly succeeded at/"solved" the ergonomics of concurrent colored-function programming for a bunch of technical reasons that Boats cal...
> consider the trait system and borrow checker, two of the most important features of Rust that were present in v0
This is quite wrong in fact, early versions of Rust were very Go-like, with a focus on green threads and GC. Even Rust 1.0, released in 2015, had a very restrictive borrow checker compared to what's available today. Rust's editions system has allowed for a whole lot of pretty seamless evolution.
Re async not being there from day one, this is indeed a big problem. it resulted in certain painful technical compromises (eg Pin) & probably at least ans importantly resulted in cultural / educational issues. The narrative around concurrency in rust initially centered how ownership & borrowing solve data races and it’s hard to see how stackless coroutines fit into that. To many users it clearly feels like an alien invasion.
Pin is painful but we're still quite not sure how the alternative (probably involving some ?Move trait) would have worked (or, for that matter, how it could work in some future edition). How do you create a !Move object when we don't even have construct-in-place in current Rust?
The definition of !Move was always the (rather baroque) "cannot be moved once its address has been taken;" this allowed you to construct them, allocate them, etc until you start manipulating them. Pin just took that contract and instead expressed it by having the value that is "the address of the !Move type" encode it.
The questions with adding Move were strictly around backward compatibility & useability, not with determining the semantics.
Boats, if you're still reading this: I don't have anything of technical substance to add to the discussion, but thanks for repeatedly explaining the rationale behind async Rust. And thanks for doing what you believed you had to do to make Rust a successful language, and especially, as you explained in an earlier post, for implementing the big feature that the companies who were most willing to fund Rust development wanted. I hope this can at least somewhat counterbalance all of the rants, complaints, and uninformed but overly confident comments.
Edit to add: I haven't personally done much work with async Rust yet, but the thing that appeals to me about async Rust is intra-task concurrency. So I'm not just trying to counterbalance the complainers and make you feel better; I really believe that Rust's futures and async/await syntax were not a mistake.
98 comments
[ 2.9 ms ] story [ 155 ms ] thread[1] https://vorpus.org/blog/notes-on-structured-concurrency-or-g...
I am deeply aware of njs’s notes on structured concurrency & have been working on a post engaging with it. This post was not about structured concurrency & so it wasn’t relevant here.
> The popular concurrency primitives – go statements, thread spawning functions, callbacks, futures, promises, ... they're all variants on goto, in theory and in practice. [...] Therefore, like goto, they have no place in a modern high-level language.
That's a really long and abrasive article ("Everyone else's stuff is GOTO, but mine isn't") to read just to get to "Wait for your return values before moving on".
No mention of any kind of synchronisation primitives, liveness guarantees, determinism, channels, transactions...
> His claim is that structured concurrency is the only way to make Ctrl+C handling work in a way how users naively expect it to work.
> And it’s easy to see why: If the exceptions are propagated up the call tree, freely crossing the thread boundaries, the KeyboardInterrupt exception is eventually going to reach the top of the call three (main function) and exit cleanly.
> However, there’s a hidden race condition involved: Imagine two threads in a scope, one gets KeyboardInterrupt and the other one raises an unrelated exception. If the latter exception bubbles up to the parent thread faster it will cancel the thread that’s processing KeyboardInterrupt and Ctrl+C would get lost.
I wonder if progress has been made since 2019.
I don't think there's a race condition in what you've described. Trio and that article is about single threaded async applications, but it's true that it could apply to threads in principle. But, in that case, I would expect cancellation to only be propagated at specific points, much like Trio's checkpoints [1]. You can see exactly that concept applied to threads in C#'s CancellationToken object [2]. You can only cancel a C# Task (which can be running in another thread or asynchronously in this thread) by using one of those, and the cancellation only takes effect when the Task checks it (e.g. by passing the token to a suitable async routine, or by explicitly calling CancellationToken.ThrowIfCancellationRequested()).
So, in the example you gave, there's two possibilities: (1) A race happens because an exception is thrown right after a checkpoint (i.e., cancellation check) and the cancel exception gets thrown at that checkpoint - but that's not really a race (except in your application logic) because that's what you asked for. Or, (2), you get to the point of throwing an exception in one thread after cancellation has been requested from another thread (but before you got a chance to check it) - but in that case the exception will propagate to the top of the thread's call stack (if not otherwise caught) and out of the thread into your nursery-like concept. So you'll end up with a multi-exception containing the KeyboardInterrupt and the other thread's exception.
In the second case, there's a "race" in that you requested cancellation of a task but it finished before it had a chance to be cancelled. But that's always a risk, even with single-threaded async applications. I think it's just fundamental to the concept of cooperative cancellation.
[1] https://trio.readthedocs.io/en/stable/reference-core.html#ch...
[2] https://learn.microsoft.com/en-us/dotnet/api/system.threadin...
(Aside: Personally, when using Trio in any application, I run it in a secondary thread, and I save the main thread for just waiting for KeyboardInterrupt (or for the secondary thread to finish naturally). That way, I can propagate the cancellation to the right bit of my application, which allows shutdown in an orderly fashion. I view Trio's default KeyboardInterrupt handling as just a decent attempt for those that aren't writing a serious enough application to need really reliable handling.)
func yield[G AnyGenericEvenInterface](promise G) G {...}
I'd have preferred the language expand yield() to take an argument which yields until the ((promise)d) variable is resolved to a value. This makes it more clear that the work is being passed off to runtime and will wait for that value.
I'm still somewhat confused that promise constructions don't have any well defined start time https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... They could start instantly or defer entirely until waited on by any of the resolution methods. It seems more complex than light threads / goroutines synchronized by communications queues and other first class language objects.
Does anyone have some thoughts on this? I just lack the knowledge to really evaluate that sort of take. There's a lot of discussions on whether async Rust is "good" or "bad", or the right abstraction or not, but it's pretty clear that (at this point) we're more less stuck with it (for better or worse), but that's not what I am asking about.
What I am asking is whether a proper effect system would have genuinely, and legitimately improved or significantly simplified the whole state of Rust async or not, and how. A lot of the async Rust problems sound like "hard" problems, and I'm a bit skeptical that a feature like that would really reduce the "heavy lifting" you have to do.
Discussed most recently @ https://www.abubalay.com/blog/2024/01/14/rust-effect-lowerin... / https://news.ycombinator.com/item?id=39005780 It's a viable approach if you can take the time to get it right. Async rust started out with more of a limited, MVP approach, where they first did the simplest thing that would surely work, and then went on from there.
It does sound very appealing, but I'm not sure there's enough prior art to steal from, even now, but especially not when Rust was hatching. It would probably be too big a gamble. I'm checking out Koka in that space, but it's early days.
This is not uncommon in big-world games. I hit it in my metaverse client. It's a standard Windows feature. Windows file systems have "FindFirstChangeNotificationA", which lets a process monitor a collection of files for modifications. That has scaling issues for large numbers of files, but it's the same problem. Similar ideas exist in the database world. You'd like to be able to ask "tell me when the results of this SELECT change". That's been played with as a concept, but didn't go mainstream.
In this area you can turn a manageable special case into a really hard generic problem. This is useful if you need a thesis topic, less useful if you need a working system.
To be honest, the system that we designed wasn't that dissimilar from promises. It was a dynamic collection of callbacks (using built-in C# events), not unlike then/catch delegates. I've worked on it about 10-12 years ago, exactly when JS world had discussions on what flavour of promises was the best, and it was one of the main sources of inspiration.
* Object wants some asset.
* Asset might be in memory in a cache. If so, satisfy request from cache.
* If asset is not in cache, queue up request, possibly on a priority queue that gets re-prioritized. Merge requests for same asset.
* When request reaches front of queue, see if the things that wanted it still want it. Remove dead requests. Discard entire request if all requestors no longer need it.
* Lock against two asset fetches for same asset being performed simultaneously.
* Fetch and preprocess asset
* Check again that asset is still wanted.
* Deliver asset
* Unlock against two asset fetches for same asset.
This is almost generic, except that once level of detail becomes involved, it gets less generic.
I also like the point that maybe(async) isn't terribly compelling. Recently, I've been looking at building an async parser, and the sync version of the interface is pretty simple: fn parse(r: &mut dyn Read) -> Result<ParsedObj> [1]. In theory, I could slap an async on it, change Read to AsyncRead, and now I have an async version. Except it's not usefully async: the consumer of the data can't do anything until everything has been fully read in. So what you want in the asynchronous version of the API is something that looks like a stream of parse events. But that API is way too much faff for anyone trying to use the API synchronously. Or is it? If you have a high-level method that distilled the stream of output events into a simple, easy-to-use answer, then you end up with something that looks maybe(async)-able [2].
Where I think the author gets it wrong, though, is in asserting that it's statefulness that causes maybe(async) to break down. State per se isn't the problem: our underlying parser above is statefully turning a stream of input data into a stream of parse events, and this could sometimes work well with maybe(async). The problem arises when you're multiplexing many streams at once. In synchronous code, there's just no way to do that kind of multiplexing (as pointed out earlier in the blog post), while it's a pretty key design feature of async code. The trouble really comes in when you have a system whose outer interface is a 1:1 stream interface, but which internally needs to go through a 1:N piece and an N:1 piece. It looks like it could be maybe(async)-able from the outside, but its implementation is hopelessly not maybe(async)-able.
[1] Honestly, a lot of parsers go a step further and just read in the input buffer as a &[u8] and rely on zero-copy, but that interface turns out to be even less useful in an async context.
[2] But the faff remains if you can't resort to one of the pre-canned methods. How much of an issue that ends up being is left to the reader to decide.
This seems... a bit specific? Was it a reference to a specific new language or was that more of a poignant wish?
Not very new, seeing as Erlang was open sourced by Ericsson in 1998. (The language itself is from 1986 but was proprietary up to that point.)
> You might go do that, in a less than optimal world.
Eh bien! mon cher Pangloss, lui dit Candide, quand vous avez été pendu, disséqué, roué de coups, et que vous avez ramé aux galères, avez-vous toujours pensé que tout allait le mieux du monde? Je suis toujours de mon premier sentiment, répondit Pangloss; car enfin je suis philosophe; il ne me convient pas de me dédire, Leibnitz ne pouvant pas avoir tort, et l’harmonie préétablie étant d’ailleurs la plus belle chose du monde, aussi bien que le plein et la matière subtile.
"I have always abided by my first opinion," answered Pangloss; "for, after all, I am a philosopher, and it would not become me to retract my sentiments; especially as Leibnitz could not be in the wrong: and that preestablished harmony is the finest thing in the world, as well as a plenum and the materia subtilis."
Can anyone (withoutboats?) elaborate on this? If “stackful” coroutines means functions with an ordinary-ish stack that can yield, then those stacks are just memory, and you can take references to them. Heck, you can do this, slowly and painfully, in C, with setcontext. And you can mostly do FFI with a degree of care about stack sizes. You can even yield across FFI boundaries. In some sense, the coroutine objects implicitly created by Rust and C++ are really just stacks minus the hardware stack pointer part.
Also, I found this amusing as a kernel programmer:
> the implementations of each of these things are completely different. The code that runs when you spawn an async task is nothing like spawning a thread, and the definition and implementation (for example) of an async lock is very different from a blocking lock: usually they will use an atomics-based lock under the hood with the addition of a queue of tasks that are waiting for the lock. Instead of blocking the thread, they put this task into that queue and yield; when the lock is freed, they wake the first task in the queue to allow it to take the lock again.
When blocking a thread, Linux (and most conventional operating systems) put the thread into a queue, mark the thread as non-runnable, and yield, and when the thing they’re waiting for is ready, they mark one or more queued threads runnable so the executor, I mean scheduler, will run them so they can try again :)
Most OSes do this a lot more slowly than can be done in userspace.
See http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/p136... for an in-depth analysis. The short version is that stackful fibers turn out to be a terrible choice. The author (Gor Nishanov, for the C++ ISO committee) explicitly says that they should not be used - this is even in spite of the fact that Windows API explicitly supports fibers as part of the programming interface, at a far deeper level than just POSIX setcontext.
Call stacks are just a data structure like any other. That they happen to be managed by the compiler to store things like local variables doesn't change this. Consider that there are algorithmic ways to convert from function calls to this data structure (defunctionalization) and back (refunctionaliztion), and the correspondence between recursive programs and iterative programs using an explicit stack.
The question here is, for any given stack, what requirements does it place on the memory layout and allocation for that stack? Gor's analysis covers a few of these, but not everyone has the same requirements and not every implementation style makes the same tradeoffs.
And you can mix and match these styles within a single programming language! Rust async functions and C++ coroutines both still store much of their local state on the native C stack- they only use their state machines to store local state (including the state machines of their callees) that lives across suspension points. This is what enables Rust to support APIs like spawn_blocking or block_on- you can run different parts of a program using different styles of stacks.
Call stacks are a part of the system API/ABI; your fancy ersatz stacks not so much. This is kind of a key consideration in the above analysis. The whole point of having Windows API support for fibers was to try and obviate that basic pain point, but ultimately this has been shown not to be very successful.
> Rust async functions and C++ coroutines ... only use their state machines to store local state ... that lives across suspension points.
Yes and this is briefly mentioned in that analysis as a key advantage of stackless coroutines compared to fibers. If you might call a function that requires 500k of stack, a fiber has to provide for that amount of stack space to be available; a stackless coroutine doesn't need to store the 500k of data, provided that it doesn't live across await points.
The reason Windows fibers failed is not because they were stackful, but because they tried to use only one stack at a time.
As long as you have that, it can be segmented, or it can be contiguous and movable for growth if you have a GC and/or forbid pointers to locals, or it can be perfectly sized if you do a whole-program analysis on stack usage, or it can be large and non-moving like the C stack.
And you’re right that the mutex in the kernel is implemented with a similar algorithm - I just meant that the code in the library you’re using is very different, because it is a relatively thin wrapper around the relevant syscalls whereas the async one is implemented in userspace.
Everything else about “why can’t I mix these functions” is a direct result of that broken consistency exposed by the OS. As such I’m not sure there is a zero-cost abstraction (stackful coroutines is close) that any programming language can build to improve the situation.
Meanwhile, the trend is clear - every OS has adopted (and is adopting more) non-blocking syscalls because they are direly needed. The only benefit blocking syscalls offer is sugar to improve syscall ergonomics.
I think if more people talked about it this way it would become clear that adding a “blocking” tag to the syscalls and bubble that tag up the call stack is the right next step to depreciating those legacy OS APIs. I don’t mean to say we should accept poor ergonomics but adding the blocking tag is a great first step to 1. Reminding people of the problem. 2. Identifying areas where research is needed to improve ergonomics and replace the existing “blocking” tag with minimal downsides.
Indeed, async v. sync should be a matter of the application and the application only.
My short summary would be in languages that result in function-coloring due to async/await, using Future/Promise directly avoids the coloring and the ecosystem is better off.
Places where async/await are ok are JS (because both the browser and node/deno) are for the most part in an async context, and Go since everything's a goroutine hence all one color.
Languages such as Java have worked well with Future/CompletionStage and doesn't really need the convenience of writing async/await which translates the rest of the program after await into a hidden 'rest of program' callback. That async/await syntactic convenience is not worth the complexities added to libraries. Rust including async/await was a poor 'cargo cult' choice. I would welcome a Rust fork/ecosystem without it.
If you're a language/platform like Swift or Rust (see also Erlang or OCaml) you're gonna end up with some kind of userspace task scheduler because OSes don't give you enough control over threads. If you're a systems language you're gonna use stackless coroutines because speed/memory and C interop. In Rust, you use their task scheduler via async/await, but sadly the ergonomics aren't wonderful and there's now weirdo problems like what reactor/engine/whatever they standardize on etc. The interesting and good stuff will be built on top of async/await, probably shouldn't have touted it as the interface.
My hypothesis here is that something like concurrency/parallelism exists on two levels. You've gotta get the underlying implementation right, and you have to expose a mental model and affordances for using it. The problem is that languages that are good at the first thing are usually real bad at the second thing. More broadly I think this is a trilemma--pick any 2 for your language:
- small
- can implement performant concurrency/parallelism
- can express a high-level mental model of concurrency/parallelism
No one should be surprised that Rust will throw "small" overboard here. I think the only question is how many models will they support? Will they do CSP AND actors (they're different!)? Will there be some kind of OpenMP type thing? Will there be new keywords or--shudder--new SYNTAX?
---
I feel like I have to say Go's solution to this is bold and elegant. Giving up straightforward C interop was pretty gutsy, and the affordances for working with its task scheduling system are familiar and intuitive. I want to stress this (because TFA goes way out of its way to be a little shitty w/ Go): this worked absolutely great. Tons of good software is written in Go. It is super effective. It spent its innovation tokens on experience and tooling and that was really successful (i.e. type systems aren't the only way to fight data races). There were exactly zero debates about stuff like await syntax or what engine to use/standardize. In the time it took Rust to ship async/await, probably 7 billion actually useful Go programs were shipped. Just because someone else's values aren't yours doesn't mean they're the wrong values. I think it's cool that people like Boats are being really thoughtful about how to do this and sharing their thoughts and processes out in the open. I just think there's room in this crazy world for at least two points of view, and that we could do with a little less sneering.
Rust builds support for all of these models based on it's basic underlying "shared ^ mutable" and "Send/Sync" safety guarantees. It should be quite clear based on this whether you're sharing access to some read-only data, staying within a single-threaded context with mutability, or relying on some sort of synchronization primitive (be it atomic data, locks, r/w mutexes etc). The fact that we understand some uses of these facilities as involving patterns such as "CSP" or "Actors" is quite beside the point.
> We should aspire not to simplify the system by hiding the differences between futures and threads, but instead to find the right set of APIs and language features that build on the affordances of futures to make more kinds of engineering achievable than before.
The stuff you listed, along with async/await, makes this possible. Isn't that what TFA is saying?
1. The words are completely awful. There's nothing "not synchronous" about an async function. Maybe at best "not synchronized".
2. The idea of pretending async functions are still "functions" is kinda terrible.
This is really a huge problem in say python where the architecture (iirc) is via iterator/yield, so your function "actually" has multiple outputs.
Maybe a radical idea in PL design would be to actually make "async functions" be some sort of other thing in the same sense that, for example, pascal had "procs" and "functions"
You could easily solve the coloring problem by creating some sort of builtin that does an immediate, blocking execution of the function by compiling out all the suspend sites, or keeps them "in place" in the case where you want to keep things as if they are futures.
This makes no sense to me. An async function is just syntactic sugar for a (normal in every way) function that returns a promise.
My novice understanding of the Rust model is that an async function is compiled down to a state machine and a set of functions which execute parts of the async function's code and then manipulate the state machine. Those manipulations have two broad categories of side effects: they can wake up/be observed by pollers, and can change the state of later parts of the task corresponding to an invocation of the original async fn (e.g. by setting variables for subsequent functions in the set--code on the other side of an internal await point that uses the result of the await).
In other words, async functions in some languages are more than just sugar for "delegate to event loop and return promise resolved by delegate completion".
So every time you call foo(); you anticipate that it might resolve immediately, or it might take some time, but it won't block any other function (unless it actually does some cpu bound operation)
And any time foo(); does something asynchronous, all of its callers, and its callers callers, become implicitly asynchronous too.
Of course you would have to have some primitive for when you actually want to do several things concurrently within the scope of a single function rather than blocking that function, but that doesn't sound too bad and in JavaScript you effectively need to use Promise.all most of the time anyway.
I'm sure there's some major downside I'm missing -- but what is it?
Even is your language run time is fully async, it's really hard to do anything useful without touching C/C++/the kernel/system libraries, and as soon as you do that all those guarantees go out the window, because you cannot rewrite the world in $newlang.
Where this breaks is with FFI: if you cannot intercept blocking calls in foreign functions, you block useful threads or even deadlock. This is what the quote in the article is about:
> The cost for the native compatibility with the C/system runtime is the “function coloring” problem.
this is relevant because miller explains why he designed promises the way he did, although since he wrote the dissertation in 02006, he couldn't yet explain how he adapted them for incorporation into the javascript standard. following his dissertation, he was on the ecmascript committee for a number of years, which is how we got e-style promises in javascript, although obviously that also involved the committee as a whole being convinced that they were a good idea. (much of the dissertation is actually about object-capability security, but chapters 13–19 are about promises.)
the great benefit of promises is that you eliminate data races, where one procedure is correct only if a particular piece of mutable data remains unchanged, while another concurrently runnable procedure changes it, so the interleaving of their accesses can result in a failure. consequently, you don't need locks or queues to communicate between different concurrent procedures; you can just use a shared data structure with no locking. this is because promises are an alternative to threads. if you're familiar with async rust, you will note three glaring problems with this:
1. rust eliminates data races through the type system, so the great benefit of promises doesn't exist in this context
2. tokio runs multiple tasks concurrently, so if you do have data races, promises won't save you
3. so you do in fact use channels and locks in async rust
so i am puzzled about why you would ever want to use promises in rust, and i guess this post is an attempt to answer that question. it explains why people have this bizarre misconception that promises are just an efficiency hack, because in rust that's all they are, and those people have never used promises in javascript or e or the other children of e such as spritely goblins.
this bears repeating. before reading this post i didn't realize this, but rust async/await is just an efficiency hack. even the section of this post entitled 'i don't want fast threads, i want futures' never brings up, as far as i can tell, any correctness or flexibility advantages for async rust, only efficiency advantages ('to multiplex arbitrarily many perfectly-sized tasks on a single thread', 'much better performance characteristics' etc.) there's brief lip service to concision ('concurrency-related boilerplate about spawning threads', which i don't think actually exists) before returning to the main theme of performance. then it proposes some truly astoundingly terrible ideas about automatically introducing race conditions if you await with a lock held, in the name of being 'even more optimal', at which point we're well into 'mongodb is web scale. if /dev/null is web scale i will use it' territory
a terminological quibble: i wish people wouldn't refer to promises as 'futures'; a future is, or anyway originally was, a different approach to the concurrency problem, one where reading from a perfectly normal variable would block your thread until that variable's value had been computed (allowing other threads to run). this approach does not have the advantages of promises. promises are what async rust implemented, just under the name of 'futures'. really, though, confusing terminology seems to be the least of the problems with rust's implementation of the paradigm
I don't know about this one. Programmers who use async Rust often talk about how they appreciate that `await` is explicit because it lets them coordinate state using cooperative scheduling. This is maybe a bit higher-level than just data races, but it is about avoiding certain kinds of race conditions.
> it explains why people have this bizarre misconception that promises are just an efficiency hack, because in rust that's all they are
I'm not sure about this either. The post argues that they are more than an efficiency hack in Rust, too- that intra-task concurrency is a useful control flow structure and promises are what enable it. That intra-task concurrency happens to share a thread is a performance benefit but that's incidental to its flexibility benefits.
> promises are what async rust implemented, just under the name of 'futures'.
Well, it's not like Rust's promises are exactly like JavaScript's promises either. JavaScript promises are continuation/callback based; Rust's are poll-based. In a sense they're like your earlier definition of "futures," except that the future API lets its caller decide how to block when the result is not yet available.
i don't agree with your implication that data races are a low-level thing. they can be, but the canonical example of a data race is a lost write to a bank account, which is not just layer 7 (the application layer) or layer 8 (the user) but layer 9 (the economic production system that employs the user)
> JavaScript promises are continuation/callback based; Rust's are poll-based
this is obviously important for the implementation of a so-called async runtime such as tokio, but as i see it, from the point of view of most of the user code, this is an implementation detail. i was talking about the semantics promises provide to things like an http client, not how those semantics are implemented
> the future API lets its caller decide how to block
it's not clear whether in this phrase you are referring to the real future api or to what rust confusingly calls 'futures'. in case you mean the former, i want to clarify that, in the futures interface defined in 'multilisp: a language for concurrent symbolic computation', (halstead, 01985, https://dl.acm.org/doi/pdf/10.1145/4472.4478), you cannot distinguish a future from the value that it eventually produces, and therefore the caller cannot decide how to block; it doesn't even know it's blocking. as halstead explains:
> An operation (such as addition) that needs to know the value of an undetermined future will be suspended until the future becomes determined, but many operations, such as assignment and parameter passing, do not need to know anything about the values of their operands and may be performed quite comfortably on undetermined futures. The use of futures often exposes surprisingly large amounts of parallelism [now conventionally called 'concurrency'] in a program, as illustrated by a Quicksort program given in Figure 1. (...)
> These “futures” greatly resemble the “eventual values” of Hibbard’s Algol 68 [39]. The principal difference is that eventual values are declared as a separate data type, distinct from the type of the value they take on, whereas futures cannot be distinguished from the type of the value they take on. This difference reflects the contrast between Algol’s philosophy of type checking at compile time and the Lisp philosophy of run-time tagging of data.
i would claim that, since halstead's paper has been cited 1658 times in google scholar, including 25 times in the last year, his definition of 'future' is very much in current use, and so it is sort of an act of intellectual vandalism to promote a conflicting but confusingly similar definition
I don't mean low level in that sense, just that they're a narrower concept than general race conditions.
> it's not clear whether in this phrase you are referring to the real future api or to what rust confusingly calls 'futures'.
I'm referring to Rust's API.
i'd still be delighted to see examples of how you can coordinate state changes with 'await'. git repositories containing purported examples would be fine, i'm not suggesting you write an entire async rust program in an hn comment
from my point of view, at this point, rust async seems to be a terrible design blunder. rust's design for massive concurrency is statically proving race-freeness, and that's what makes rust such a promising development. async in rust seems like the kind of development that could destroy the community and render the language worthless, while simultaneously torpedoing public understanding of halstead's futures model (though i don't think it's very promising) and javascript's async approach to concurrency (which is), ruining their chances to succeed as well. so i'm highly motivated to investigate any evidence that i'm wrong
What an extreme take!
Async Rust as-it-is makes perfect sense for Rust's primary audience: folks who desire control/performance enough that they would otherwise be using C/C++. To call async an "efficiency hack" is to underestimate how important that efficiency is to Rust's existence. If async didn't work the way it does, it wouldn't have gained traction within the Rust community (folks would have just ignored it and kept writing state machines by hand) and it wouldn't have been able to act as a draw for C/C++ developers. It's a killer feature, and has enabled the language to thrive.
Not only is it a killer feature, the way it is designed is the only way it could have been. See: https://without.boats/blog/why-async-rust/
async in rust is the opposite, and to me it looks like such an expensive way to get that efficiency that it strongly undermines the reasons for adopting rust
I don't think that's charitable. While there's not, like, a concise bulleted list of async's advantages other than performance, many of those advantages are discussed throughout the piece:
- Cancellation (and the corollary, the ability to externally control, suspend, or time out arbitrary sets of computations at yield points).
- Ergonomic (rather than just performant) heterogenous multiplex-waits. Doing that in non-future-oriented systems isn't just potentially slow, it's fiddly and a breeding ground for bugs in many languages, including Rust.
- The ability to outsource the geometry of concurrency and parallelism to an async runtime rather than spending brain and editor cycles deciding things like e.g. how many handler vs. acceptor threads you're going to manage, and whether executors will launch/stop lazily or eagerly.
Those aren't necessarily unique to Rust, but they are definitely discussed and are definitely unique benefits of an async/promise-based concurrency system.
Also, minor nit:
> tokio runs multiple tasks concurrently
I think you mean "in parallel" here; the whole meta-system is concurrent, but data races re-emerge in Tokio (or rather, impose sizedness/pinnedness restrictions on data in async contexts in Tokio) where they aren't in JS because Tokio can run in true threaded parallel.
> - Cancellation (and the corollary, the ability to externally control, suspend, or time out arbitrary sets of computations at yield points).
i'm not convinced by this one; i'd need to see more details, but my intuition is that it's just an efficiency hack, and i'll explain why
in a shared-nothing system like erlang (or pre-threading unix) you really can externally control, suspend, or time out arbitrary sets of computations. if you have to wait for a yield point, you only sort of can; a task stuck in an infinite loop that therefore never yields will not get a chance to be canceled. moreover, if a task is holding a lock when it yields, that could be either because its own correctness depends on some shared mutable data structure staying consistent, or because it currently has such a structure in a temporarily inconsistent state that it needs to restore to consistency
in the second case, how do you cancel it? you need to somehow restore the consistency, and nobody outside the task has the information required, and if it just releases the lock and exits instead of continuing, the next task that depends on the inconsistent structure will probably crash. so you can't cancel it at the next yield; you have to wait until the task yields without any locks held. and similar remarks apply for suspension, except that the potential problem is not crashing but a deadly embrace, a particularly pernicious kind of deadly embrace because it's hard to tell who was responsible for unsuspending the task
but let's suppose that being able to sometimes cancel, suspend, or time out a computation is a useful feature. we must ask, then, when is it useful? if a task is computing a result that is no longer needed (perhaps the user has already been shown a 'timed out' error, for example) what is the harm of allowing it to continue as long as it wants? it depends on the task, of course, but it had better not be crucial to correctness, for two reasons. one is that, as explained above, the cancelation might have to be delayed for an arbitrarily long time so that the task can suspend while not holding any locks. the other is slightly more subtle: if the task's correctness depends on it getting canceled early enough, then its correctness is dependent on the other task that kills it getting scheduled early enough, which is generally not something that can be guaranteed
consequently, i believe the only case where task cancelation can be useful in a shared-memory system like this is when it's just an efficiency hack and doesn't actually affect the system's semantics
but maybe there's something i don't understand about this (my ignorance of async rust is truly vast) so please let me know. git repository urls containing purported counterexamples would be especially welcome
> - Ergonomic (rather than just performant) heterogenous multiplex-waits. Doing that in non-future-oriented systems isn't just potentially slow, it's fiddly and a breeding ground for bugs in many languages, including Rust.
it's possible i'm not understanding what you mean; can you explain with code? it sounds like the kind of thing you could put into a library function in a system using multithreading; you can solve it without reconsidering your entire programming paradigm
> - The ability to outsource the geometry of concurrency and parallelism to an async runtime rather than spending brain and editor cycles deciding things like e.g. how many handler vs. acceptor threads you're going to manage, and whether executors will launch/stop lazily or eagerly.
this one is purely an efficiency hack; in a threaded system you can spawn a new handler thread for each request which terminates when the request is done. maintaining a thread pool is itself purely ...
From a certain perspective, the knowledge that suspension/cancellation can only occur at yield points shrinks (or at least makes somewhat more vislble) the risk surface of areas where arbitrary cancellation can leave the program in a corrupt state.
I do agree that more work is needed in this area to ensure automatic state reconciliation after cancellation (the mutex-held-during-await issue). Python makes a decent stab at this: its async/await concurrency implements cancellation as an exception raised at await points, allowing cleanup to occur before proceeding with cancel. Unfortunately, that comes with the consequence of allowing routines to choose to defer cancellations, or ignore them entirely. Perhaps that could be mitigated with an async/await system that makes cancellation a bidirectional exchange, changing observable behavior at the canceler when the cancellee does or doesn’t actually terminate? I think the Rust folks are moving in this direction with async-drop, though I personally do not like the idea of implicit custom destructors in the first place, and worry that allowing them to effect async cancellation control flow will hurt rather than help. That’s a separate topic, though.
All that said, I think that struggle is a somewhat inevitable outcome of a platform that supports shared memory between routines and C interop. If you’re willing to give up on those, then sure: you can preempt or cancel with much more flexible semantics. But I don’t think that’s innately better: first, C interop is critical for some platforms, as Boats mentions. Second, while you might be right in pooh-poohing some of my other points as “just efficiency”, adopting an Erlang memory sharing model is really expensive—impactfully so for a much wider variety of programs than the ones for which, say, stackful vs. stackless coroutines make a difference.
On to your next point: how important is yield-point cancellation, really? I think that
> the only case where task cancelation can be useful in a shared-memory system like this is when it's just an efficiency hack
isn’t correct.
Firstly, “efficiency” is kind of a moving target. Is it merely efficiency if we can, say, take advantage of the fact that memory used by cancelled coroutines is freed once they’re cancelled? Consider this code:
https://gist.github.com/zbentley/f45ea39843f533f5f54b23f8a49...
If cancellation of the first call to memory_hungry() did not trigger the drop of x (90% of system RAM), then we’d be unable to successfully call memory_hungry() a second time after the select!, since that would put us into OOM.
Like, sure, there are all sorts of ways to accidentally leak resources in the event of cancellation (we could mem::leak it, could have memory_hungry() spin and block the task, and so on). But this is one fairly happy path where a very important resource (memory) is correctly managed by the cancel-at-yield model.
Secondly, what about the cancellation of routines that keep something alive? For one example, consider routines that send heartbeats or liveness probes: cancelling those can be very meaningful to the program’s semantics. If your rejoinder is that that’s a bad idea because of the chance of a while-true loop blocking routine causing a spurious event loop hang, I’d respond that we’re either now discussing a whole different set of constraints (RTOS behavior...
1. I updated the gists a bit for clarity, hopefully without accidentally moving any goalposts.
2. In addition to the tradeoffs mentioned in the sentence "If those aren't priorities for you...", I didn't really respond directly to your points about loop-blocking causing event loop hangs.
The possibility of loop-blocking causing the entire concurrency system (rather than just, say, one request's routines) to malfunction is a real drawback of async/await systems. Whether or not it's a deal breaker for your using such a system boils down to trust, I think. If your program is running a lot of background routines whose behavior you don't trust to have a low likelihood of loop-blocking, then async/await might be problematic for you. This is analogous to deciding whether to include a piece of functionality as a library linked into your main program, or to include it as a subprocess/IPC interface instead. If you don't trust that functionality not to e.g. corrupt memory, you may choose the latter.
I think Tokio specifically is better at mitigating those drawbacks of async/await concurrency models than many other systems, due to its abilities to a) provide fine-grained and accurate detection of loop-blockers, and b) to move task execution between threads to get around potentially stuck routines. Those are just mitigations, not solutions, but they're substantially better than what's offered by e.g. JavaScript.
Promises don’t prevent data races. They only don’t exist in JavaScript because the runtime is implicitly single threaded.
if you want to select on multiple promises, it's relatively easy to do in javascript; you make a new promise for the disjunction of the multiple promises (e.g., using .withResolvers()), and attach a callback to each of the original promises that resolves the disjunction promise when it fires
this of course does not prevent the other promises from getting resolved, because that would happen in some other piece of code, perhaps in response to a network i/o event or a filesystem i/o event; knowing whether a particular turing-complete event handler is going to resolve a particular promise (so you could avoid running it) would require solving the halting problem
It does not if you control the implementation of the language primitives ;) I’m not smart enough to answer exactly how, but Andy Winford explains how CML does it using the notion of optimistic and pessimistic stages. https://wingolog.org/archives/2017/06/29/a-new-concurrent-ml
fundamentally one of the major things promises are useful for is waiting on network packets which may never arrive. (in the small, sometimes the network in question is sata or usb, but that makes surprisingly little difference.) there's really nothing you can do to keep some other host from sending you a response packet to a request you've already sent it, and when your network interrupt fires, unless you've brought the network interface down entirely, your interrupt handler needs to do work to buffer the packet (if necessary) and inspect it to see where to route it to. if that turns out to be a promise, all of this is 'making progress on' a promise that may be canceled or whose fulfillment may have no effect. (you could, however, limit the resources you dedicate to such useless work: an important optimization in many contexts, but not one that needs language-level support)
so even controlling the implementation of the language primitives doesn't help
I think selection is about what is visible to your program. I.e. that even if that packet arrives, if you choose another path in the select, then the recv() is guaranteed not to happen. You are right that a program doing this has to handle this case correctly, which I think is what the whole Rust future cancellation conversation is about.
I don't really understand this point.
In a hypothetical world where
(a) threads are cheap, and
(b) threads support some form of cancellation (with an ecosystem that expects this and is built around it),
then you don't need "intra-task concurrency" as a fundamental primitive. You can achieve the exact same semantics by just having a helper function that, given N closures representing work to perform, spawns a new thread to run each one, and waits for any/all of them to finish, while making sure to cancel all those threads if it itself is cancelled. (Or if you want to manually manage future objects for more control, you can do that too. But a future just represents work being done on a thread.)
Compare to Go. Go has cheap threads. Boats criticizes Go's "threads, locks and channels", but the fact that you have to interact with those primitives directly, rather than using a helper function as described, is really a limitation of Go's type system, not an inherent result of basing concurrency on cheap threads.
With one exception: Go threads do not support cancellation. Cancellation is extremely important! Without it, you have to manually pass down timeouts and other forms of cancellation into each function that performs I/O, and it becomes impossible to write that kind of generic helper function.
And of course, Rust has to fit into an existing thread ecosystem that does not meaningfully support cancellation, and does not make threads cheap.
So the tradeoff for Rust remains. Still, I don't see this anywhere near the way boats sees it. In my ideal world, if I didn't care about performance or compatibility, I wouldn't want "a language in which all functions are coroutines, meaning that all functions can yield." I don't think yielding needs to be visible to the user in any meaningful way; why does it need to be? It can be a hidden implementation detail, just like it is with green threads or OS threads. But I'd add on a concept of cancellation.
…In short, I think my ideal model is something like Trio, but I haven't had the opportunity to actually use Trio. I hope that changes.
(edit: and I'm looking forward to when boats does respond to the Trio blog post.)
As an aside, when folks mention algebraic effects in statically typed languages, delimited continuations and effects are equally powerful (possibly identical?). IMO effects seem to be about type safety and static typing, which does allow some neat optimizations.
> I don't think yielding needs to be visible to the user in any meaningful way; why does it need to be?
It's an interesting question. Users definitely want coroutines they can yield themselves - this is was iteration is all about, for example, and there are other similar patterns (the entry API is ultimately a kind of coroutine).
At the same time, the highest performance low level interfaces for asynchronous IO are ultimately just asynchronous iterators & asynchronous sinks over shared memory ring buffers, io-uring is the most famous example but there are others like AF_XDP.
I think you're right that a kind of cancellable virtual thread ultimately would have analogous affordances to a future. But a future is just a specific class of coroutine (that yields a unit type), so if you will have the coroutine mechanism for the rest of the system, why would you not extend it to include IO instead of adding a separate mechanism to erase the fact that IO is happening? What you also get from this is that if a function yields the bottom type you know it doesn't perform IO.
It definitely makes me wonder if the original sin of Rust and many of the other colored-function-based concurrency systems was limiting the enforcement of color-can-only-call-color to only async functions. Would a language that disallows calls back from async functions into blocking functions make sense? Probably not. But what if we split "blocking" into two categories: functions that provably blocked for constant time (or even constant-less-than-X time; simple systems like BPF can verify that already, so I'd assume it's possible to get more nuanced here with effort) versus functions that did not (including I/O, while-true-with-non-statically-verifiable-condition)? If we disallowed calls from async functions into functions of the latter category but not the former, would that be ergonomic/accessible for programmers?
(A minor quibble/criticism of an otherwise great post: I think the discussion of "blocking functions" should get a crisp definition up front, qualifying that "blocking" in this case means undesirably long or potentially indeterminate blocking of executor threads, so readers don't get confused and wonder whether the implication is that any synchronous function call is inherently--rather than just potentially--toxic to good async behavior).
Continuing to think about the idea of "async code can't call 'bad blocking' code": I suspect that one of the reasons this isn't widespread is that it really does require compiler/runtime validation that code down a call stack is good/bad. Erlang almost got there, but (just like Boats points out happened with Rust) the C FFI chimera stopped them in the home stretch and they settled for asking users to tag dirty vs. clean NIFs. But maybe compiler awareness of call path blocking risk is advanced enough today that such a system could be implemented? It definitely wouldn't be simple (...he said, as if he were anything but a rank amateur when it came to understanding compilers). It definitely would solve the mutex-held-across-await-point class of problems.
Two other points that stood out to me that I thought were worth mentioning:
1. The rejoinder to the blocking-reqwest-vs-ureq situation is well put. Honestly, Boats could have gone further here: I think that a large proportion of the complaints/reluctance around using synchronous APIs that wrap block_on or whatever are fundamentally aesthetic/emotional objections arising from people's bad vibes around including or separately instantiating an async runtime inside a library just to provide a blocking API. In practice, the compile overhead is rarely severe and nearly always paid extremely infrequently, and the runtime overhead in time/memory is negligible in release builds. Supporting the emotional resistance here is, I think, a lot of user ignorance around the practical differences between, say, how a current_thread Tokio entry point runs Rust futures and how a libuv loop or a boost::asio context or whatnot run futures in JS/C++/etc. While Rust and Tokio aren't magic (and while I would love more crates to be async-to-syncable with pollster alone), I think that folks often project assumptions about heaviness/performance from other systems onto parts of the Rust ecosystem where they don't hold as true.
2. The Golang callout is fair (and was hopefully cathartic to write), but in addition to being punchy I think there's a lesson to be learned there, albeit a bitter one: Go is widely considered to have wildly succeeded at/"solved" the ergonomics of concurrent colored-function programming for a bunch of technical reasons that Boats cal...
Rust async support is explicitly not complete. They've got an entire working group that's focused on improving it as per https://rust-lang.github.io/wg-async/vision/roadmap.html .
> consider the trait system and borrow checker, two of the most important features of Rust that were present in v0
This is quite wrong in fact, early versions of Rust were very Go-like, with a focus on green threads and GC. Even Rust 1.0, released in 2015, had a very restrictive borrow checker compared to what's available today. Rust's editions system has allowed for a whole lot of pretty seamless evolution.
Re async not being there from day one, this is indeed a big problem. it resulted in certain painful technical compromises (eg Pin) & probably at least ans importantly resulted in cultural / educational issues. The narrative around concurrency in rust initially centered how ownership & borrowing solve data races and it’s hard to see how stackless coroutines fit into that. To many users it clearly feels like an alien invasion.
The questions with adding Move were strictly around backward compatibility & useability, not with determining the semantics.
Edit to add: I haven't personally done much work with async Rust yet, but the thing that appeals to me about async Rust is intra-task concurrency. So I'm not just trying to counterbalance the complainers and make you feel better; I really believe that Rust's futures and async/await syntax were not a mistake.