Tasks are Rust's unit of concurrency. There are two models for scheduling tasks. 1:1 means that each task runs in it's own operating system thread, and the OS is responsible for scheduling. M:N means that M tasks run on N threads Typically, N is roughly one per core, and M is greater than N. The runtime is responsible for assigning tasks to threads and scheduling multiple tasks on a thread. "Green threads" is a term from the Java world, and it means M:N, often with N=1, regardless of how many cores are available.
The motivation for M:N is that OS threads tend to be fairly heavy in terms of memory usage, scheduling overhead etc. For a language that intended to enable highly concurrent programs, it's theoretically more efficient to use one OS thread per CPU to get maximum parallelism and layer the units of concurrency (tasks in this case), on top of that.
The thing is, doing M:N well enough to actually achieve the theoretical gains is difficult, and there are many cases of projects that started with M:N and eventually ditched it and moved to 1:1.
Rust has the concept of a shared heap meant for sharing memory between tasks. Tasks do have to claim (and then release) ownership of references from this shared heap, so this works like a locking mechanism, preventing 2 tasks from modifying a reference at the same time.
References on the shared heap can't be managed by a garbage collector. But references on local heaps can be managed by a thread-local garbage collector. Which is actually rather cool, as Rust basically sidesteps the need for having a concurrent, compacting, generational GC (in Rust the GC is also entirely optional).
This is entirely wrong. You can easily pass a raw pointer (I.e. same semantics as a C pointer) between tasks and use this to unsafely share memory (I.e. risk of data races), or you can use one of the safe higher level wrappers in extra::arc, that is, Arc for immutable shared memory, or RWArc or MutexArc for lock-protected mutable shared memory.
It's still worth while noting that Rust’s modus operandi eschews shared memory; to get shared memory between tasks, you've got to be fairly explicit about it.
The problem with M:N is that the kernel has better info on doing efficient scheduling. To make M:N work with preemptive scheduling, the process' runtime would have to communicate with the kernel and agree with it on the best scheduling strategy. And currently, this is a recipe for disaster.
This is why M:N scheduling is in practice cooperative and Rust currently does this too. How it works - basically, when pseudo-blocking I/O calls are made, the active task has to yield (by itself) the execution to another task waiting in line, while it "blocks" for the result. It's very similar to what happens in Scala when dealing with Futures/scala-async, or what happens in C# with "async", or what happens in Clojure with core.async.
Above I've said "pseudo-blocking" for a reason, because it depends on language features to execute non-blocking I/O operations as if they are blocking. The problem with this is that truly blocking operations will block the OS thread for real.
Also, this architecture is only a win if you're in a context in which your tasks are mostly CPU-bound, or if you don't have enough threads to block (e.g. like in the context of servers responding to web requests, in which the upper limit is unbounded). Blocking I/O yields better throughput than asynchronous I/O. In terms of throughput the JVM's blocking I/O primitives are beating any M:N architecture, like Golang, because yes, epoll comes at a cost. Heck, even asynchronous I/O on top of the JVM has better throughput, again because of 1:1 multi-threading.
> it's theoretically more efficient to use one OS thread per CPU
This is not true.
In case you're not talking about pure CPU-bound tasks, that CPU core will be left at less than 80% usage and you can't really have pure CPU-bound tasks (even CPU-bound tasks have to wait for I/O, even if asynchronously, which at the very least involves interactions with the kernel). This depends of course, but if you've got a CPU core that's used at less than full capacity, then there's room for another thread or two on that core. Which is why the optimal default configuration for CPU-bound tasks are thread-pools with a number of threads that's 2 times the number of CPU cores, which is a reasonable default left to be improved by real profiling.
Also, blocked threads DO NOT consume CPU resources. The real problem is when those threads wake up, participating in context switching. Which is why, if you've got threads blocking for I/O, it's not a problem if you have 100-200 threads that mostly sleep, waiting on some resource to be available.
To give you an example, on top of the JVM it's not a problem if JDBC is based on blocking I/O. You can have like a thread-pool that waits for SQL queries to execute and when the result comes in, you pass the results to another thread-pool meant for CPU-bound tasks. That's why well-grown JVM application are using at least 2 thread-pools to execute tasks - one for CPU-bound tasks with a size directly proportional to the number of CPU cores and one meant for I/O bound tasks, which is unrelated to the number of CPU cores. As an aside, doing this is also good for avoiding dead-locks from combinations of CPU-bound and I/O bound tasks interacting with each other on the same thread-pool.
BTW, there isn't a single platform that went with M:N threading without regrets. That's because you can easily build M:N cooperative multi-threading on top of 1:1 preemtive multi-threading, but this doesn't work the other way around.
Is there a way to download this? It won't buffer properly for me for some reason.
(EDIT: Also, I don't know who made that video interface, but a "HD" button where clicking does not make it obvious whether or not you set or unset the HD option and have to wait for the video to load or not is pretty terrible)
I'm curious as to what the perf figures would be for windows. My gut tells me threads are relatively more expensive and async relatively cheaper than on Linux.
18 comments
[ 7.3 ms ] story [ 55.5 ms ] threadLinks to the PRs (the discussions are worth a read):
1. https://github.com/mozilla/rust/pull/10830
2. https://github.com/mozilla/rust/pull/10965
Can someone explain to a hw guy what 1:1 is, and M:N, and green threads, etc.? And why and when you would want to use one verses the other?
The motivation for M:N is that OS threads tend to be fairly heavy in terms of memory usage, scheduling overhead etc. For a language that intended to enable highly concurrent programs, it's theoretically more efficient to use one OS thread per CPU to get maximum parallelism and layer the units of concurrency (tasks in this case), on top of that.
The thing is, doing M:N well enough to actually achieve the theoretical gains is difficult, and there are many cases of projects that started with M:N and eventually ditched it and moved to 1:1.
(Edit for clarity on green threads.)
References on the shared heap can't be managed by a garbage collector. But references on local heaps can be managed by a thread-local garbage collector. Which is actually rather cool, as Rust basically sidesteps the need for having a concurrent, compacting, generational GC (in Rust the GC is also entirely optional).
This is why M:N scheduling is in practice cooperative and Rust currently does this too. How it works - basically, when pseudo-blocking I/O calls are made, the active task has to yield (by itself) the execution to another task waiting in line, while it "blocks" for the result. It's very similar to what happens in Scala when dealing with Futures/scala-async, or what happens in C# with "async", or what happens in Clojure with core.async.
Above I've said "pseudo-blocking" for a reason, because it depends on language features to execute non-blocking I/O operations as if they are blocking. The problem with this is that truly blocking operations will block the OS thread for real.
Also, this architecture is only a win if you're in a context in which your tasks are mostly CPU-bound, or if you don't have enough threads to block (e.g. like in the context of servers responding to web requests, in which the upper limit is unbounded). Blocking I/O yields better throughput than asynchronous I/O. In terms of throughput the JVM's blocking I/O primitives are beating any M:N architecture, like Golang, because yes, epoll comes at a cost. Heck, even asynchronous I/O on top of the JVM has better throughput, again because of 1:1 multi-threading.
> it's theoretically more efficient to use one OS thread per CPU
This is not true.
In case you're not talking about pure CPU-bound tasks, that CPU core will be left at less than 80% usage and you can't really have pure CPU-bound tasks (even CPU-bound tasks have to wait for I/O, even if asynchronously, which at the very least involves interactions with the kernel). This depends of course, but if you've got a CPU core that's used at less than full capacity, then there's room for another thread or two on that core. Which is why the optimal default configuration for CPU-bound tasks are thread-pools with a number of threads that's 2 times the number of CPU cores, which is a reasonable default left to be improved by real profiling.
Also, blocked threads DO NOT consume CPU resources. The real problem is when those threads wake up, participating in context switching. Which is why, if you've got threads blocking for I/O, it's not a problem if you have 100-200 threads that mostly sleep, waiting on some resource to be available.
To give you an example, on top of the JVM it's not a problem if JDBC is based on blocking I/O. You can have like a thread-pool that waits for SQL queries to execute and when the result comes in, you pass the results to another thread-pool meant for CPU-bound tasks. That's why well-grown JVM application are using at least 2 thread-pools to execute tasks - one for CPU-bound tasks with a size directly proportional to the number of CPU cores and one meant for I/O bound tasks, which is unrelated to the number of CPU cores. As an aside, doing this is also good for avoiding dead-locks from combinations of CPU-bound and I/O bound tasks interacting with each other on the same thread-pool.
BTW, there isn't a single platform that went with M:N threading without regrets. That's because you can easily build M:N cooperative multi-threading on top of 1:1 preemtive multi-threading, but this doesn't work the other way around.
https://air.mozilla.org/rust-meetup-december-2013/
(EDIT: Also, I don't know who made that video interface, but a "HD" button where clicking does not make it obvious whether or not you set or unset the HD option and have to wait for the video to load or not is pretty terrible)
Would appear to be a direct link (you'll probably want to right-click, save as). Sorry if this is uncouth.
For this particular video: