Hmm feels like there is a bit of a misunderstanding on the meaning and cost of threads and how the OS schedules work from threads onto logical CPUs and what state they are in. It is expected that processes might use more than 1 thread, and it is expected that you would have more processes than logical CPUs. Can not-dead process exist without a thread at all? I'd expect not.
7000+ threads does seem like a lot though and agree there is the general lack of caring about resource consumption in all forms, eg Electron apps. At this point having lots of threads seems almost expected by various OSs and something they solve for to reduce the cost of it so the user doesn't have to worry about it.
If all the threads are idle then you could exit all of them; this makes sense if they're using async system calls. The OS would just create a new thread in the process to resume it again.
Of course, there's a lot of tradeoffs between that and just keeping an idle thread around.
> Most threads are waiting for something – this is not the ideal activity for a thread. A thread should run (useful) code.
Seems like that is an ideal activity for a thread, and in fact why most people use them? Combining all those waits into a single thread/process means also creating a dispatcher which then either spins up a thread to do work (ahem), or blocks all other useful work. If the cost is low (and for thread it usually is), having lots of blocked threads often makes the code easier to write and maintain.
you can't have more parallelism by simply increasing #threads. Parallelism is bound by CPU cores.
Threads need to scale with CPU cores, however simply increasing #threads doesn't help. You need to split tasks of your program such that more tasks can be executed in parallel while not adding more overhead than you gain by parallelism.
Then you can distribute those tasks on as many threads as possible (while still being bound by #cores).
That isn't what Amdahl's law says. Amdahl's law is an equation describing the theoretical paralellizability of a given task. It doesn't have anything to do with systems bottlenecking on one resource vs another.
> Seems like that is an ideal activity for a thread
Agree, because spawning a new thread and killing a thread carries its own overhead so better to have the thread sleep or be paused by the OS and cpu scheduling.
Its all about getting the balance right, but I have noticed dont net apps run their gui on a single thread instead of a window(s) linked to a thread, like in a multi document interface (MDI) setup.
It is the ideal activity for a thread, but you actually don't want too many threads hanging around because each thread costs additional memory (stack + bookkeeping) and some amount of bookkeeping complexity for the OS scheduler, assuming native threads. That cost can really add up over the entire system.
In an OS like Windows, on "typical" hardware, the amount of memory per thread for stacks and bookkeeping the OS and/or userland allocate is usually negligible. In this context also remember that the reserved stack sizes per thread do not refer to physical memory but virtual memory (or "committed" vs "reserved"). A typical thread usually commits 1 page for the stack and the equivalent of up to 2 pages worth for any bookkeeping[0], etc, which would put the memory committed for the 7412 threads the author mentions in the ballpark of maybe 30-100 MiB of memory actually committed.
The OS scheduler should be good enough to distinguish between runnable/waiting (sleeping) threads, eliminating the need to even look at most threads (which will be in waiting state) until some signal arrives putting them back into a runnable state.
You only really get a problem when you have too many runnable threads (fighting over the available resources) or when too many threads keep rapidly toggling between runnable and waitable, as either case would potentially lead to a lot of expensive context switches (and the former case is an indication you're trying to do too much with the physical resources available).
[0] Tho some things may commit considerably larger amounts of stack space and/or bookkeeping space. E.g. if I remember correctly java8 commits 1MiB of stack space per thread by default, while e.g. java11 merely reserves that 1MiB.
The standard approach on Windows (and no reason also for linux, FreeBSD) is to have per-process thread pools. E.g. each process in a .net app has a default thread pool. Windows also has an async IO api that requires a thread pool of threads to handle the IO (handled in the Win32 API, so built into the core OS).
A single thread can then wait for events that would require some processing (IO or timers being the most common) and then it passes that unit of work to the thread pool to be processed. Windows even takes pains to ensure that the last thread to process a unit of work, might then also be used to process the next item in the queue of work to eliminate costly context switching, and invalidation of data/instruction cache.
Thread pools allow the best use of threading by:
- limiting the number of threads a process creates/maintains. The thread pool itself can grow or shrink as demand changes.
- limiting the number of threads that are in a runnable state by queueing work to the pool. The pool itself will be sized according to number of CPU cores, e.g. 2 * CPU cores.
The main idea behind tread pools is that any thread that's not doing useful work is a waste of OS resources, so they make sure that they are either always fed with work, or get destroyed when idle for too long. From an overall machine efficiency perspective this makes total sense.
Having hundred or even thousands of threads created and idle is massively wasteful and bad design, the OS should not be burdened with such housekeeping.
Note that Apple's GCD also works on the same principle, as does libev/libuv.
Switching between threads is expensive, so ideally the threads are never waiting unless the system is literally idle as that would imply a context switch. For software that has a lot of fine-grained multi-thread coordination, like database engines, a substantial percentage of all CPU time is spent coordinating and context-switching between threads, not actual workload. The more cores you have, the worse this issue becomes.
This is why thread-per-core software architectures are idiomatic for anything requiring maximum performance, especially on systems with lots of cores. Similarly, there is no requirement to spawn separate threads for task concurrency. The non-blocking coroutine-style concurrency common in high-performance server software supports much higher concurrency and with context switching costs that are much lower.
The thread-per-core coroutine-style is easier to write and maintain too -- the logic is almost entirely single-threaded. No deadlocks, races, etc to worry about because lock graph concurrency is complicated.
> ideally the threads are never waiting unless the system is literally idle as that would imply a context switch
It wouldn't. There will only be a context switch if the thread is woken up for some reason (eg. the operation it's waiting on compeltes). If a thread is spending a majority of its time waiting, then it's not going to be doing a lot of context switches.
> This is why thread-per-core software architectures are idiomatic for anything requiring maximum performance
That's a stretch. There are some specific real-time usecases where you want to lock a specific thread to each core, but those are extremely niche and imply total control over what is running on the system.
These architectures are used for throughput, which is much higher than classic multithreaded architectures, not real-time per se. Though they also can have very good latency characteristics if you sacrifice a bit of throughput.
This architecture has been idiomatic for high-performance server software for a solid decade. It only seems niche if you don't work in an area of software that cares a lot about performance. The performance difference is not small, 10x throughput on the same hardware is not uncommon.
And yes, it implies control of the resources your process nominally owns, though by no means the entire machine necessarily. Again, idiomatic, and software like high-end database engines have been designed this way for a very long time so it isn't a new idea. Designing software that takes full control of its resources, and does its own I/O and process scheduling, isn't magic, just different.
A classic multithreaded software architecture will never be remotely performance competitive with a thread-per-core architecture. It isn't expected in theory, and is never demonstrated in practice. Modern thread-per-core architecture has its origins in supercomputing, specifically to enable better scalability as the number of cores per CPU (and server) became greater than 2.
> No deadlocks, races, etc to worry about because lock graph concurrency is complicated.
I’ve spent the last two and a half years mainly programming in single-threaded NodeJS environments with a focus on heavy same-task concurrency, and I’ve still run into deadlocks and races, despite the lack of OS threads involved.
However, your point that context switching is expensive, and is why thread-per-core architectures are so common, still holds.
> In an ideal world, the number of threads in a system would be the same as the number of logical processors – any more and threads might fight over processors, any less and you’re not using the full power of the machine. Obviously, each “normal” process must have at least one thread running whatever main function is available in the executable, so on my system 521 threads would be the minimum number of threads. Still – we have over 7000
Seems to not account for the fact that threads will sleep while waiting for I/O.
If a thread is waiting for I/O or a mutex, it will yield and let another thread use that CPU time to do something. This doesn't work if you don't have more threads than logical cores, because then there is no job that can take over.
We've had this type of cpu time sharing far longer than we've had multicore CPUs.
As far as the operating system is concerned, a thread is a thread. Async/await is a pattern for hiding the implementation details of multitasking or faking it in languages with limited support, it still largely works the same, except you see it as the thread being yielded to rather than from in explicit threading.
Async-await uses non-blocking operations in the kernel. If blocking operations need to occur, then these can occur on a separate thread but this is rare. Most things have a non-blocking equivalent.
When you read a regular file from disk, and it's not in cache... the thread (or process) calling read() blocks, waiting for the data.
Or perhaps you're using NodeJS-style async callbacks. Guess what happens... Your async read looks like it's just using an event queue, but behind the scenes, NodeJS calls libuv which creates a thread which blocks in read(), and when read() unblocks and returns your callback is added to the event queue.
Perhaps you're using the async/await style with the Promise API for truly async goodness. Guess what happens... Your async call still ends up calling libuv which creates a thread which blocks in read(), and your await waits until libuv's thread read() unblocks and tells the Promise that it's fulfilled.
Maybe you're using shiny io_uring() - finally (it was proposed many times before) a real async system call interface in Linux! Guess what happens... Your lovely async/await calls the pristine goodness of io_uring_enter() which... creates a kernel thread which blocks in read(), and your await waits until the kernel thread's read() unblocks and reports a result.
Even with async/await languages and frameworks, and event-driven frameworks, a blocking thread is used somewhere in the stack to read from a file.
I've measured how many threads to create for optimal throughput of small reads from NVMe storage, and the fastest number of threads varies but it's generally in the tens to low hundreds, and always more than the number of CPU cores, if the data is not already in cache.
This is a complex area to tune adaptively, and Linux itself is unfortunately not ideal for this (it doesn't tell the userspace scheduler about blocked vs preempted I/O threads). Interestingly, reducing the number of CPU cores (using taskset) while keeping the same number of threads sometimes increases I/O throughput.
(Note: The above assumes Linux, not O_DIRECT with libaio, and that data is not in cache when using io_uring).
I am not sure about Linux kernel internals and I/O operations. However, on Windows kernel the filesystem (or any driver incl. networking) can complete an I/O operation in non-blocking way, it just marks the IRP (descriptor for I/O operation) is pending and complete it later in different context (i.e. in an interrupt handler even when it is in different process/thread context). Since the user mode application can wait for multiple different kind of objects at the same time (this includes networking, file, mutexes, events), I guess in theory it can really scale when thread_count = core_count.
[My journey was basically being kool-aided on various green thread variants and async and whatnot, only to realize that threads are pretty damn excellent for most use cases, and all alternatives come with subtle drawbacks.]
Do you think that io_uring (or similar) has a ubiquitous future (as opposed to niche – which there is clearly a demand for atm)? Or is combining kernel and user space scheduling is a forever-hard-to-tune problem? Also, do you see any potential improvements to traditional threading/scheduling that could be helpful alleviating some of the resource hogging issues (which are real but over-hyped imo)? Thanks.
> In an ideal world, the number of threads in a system would be the same as the number of logical processors
... no. Not if you rely on blocking system calls. Or wait for anything that isn't the CPU. Or run more than one application per CPU on your system. Or manage concurrent code flows at any level higher than source code.
Yep, that's a great goal, and I/O-intesive applications do this, managing concurrent workflows with custom user-mode schedulers.
But now you have more than one application on the machine? Or even, code from multiple vendors which all need to issue I/O concurrently? How do you integrate their work with your custom scheduler?
You need some centralized, system-wide scheduler everyone agrees on.
Does it make sense to ask for a system wide scheduler that manages tasks (tasks in terms of the state machine behind async await) instead of threads?
Currently every app that uses tasks has it's own executor with a corresponding thread pool, which may be more efficiently used if it was shared by using a single async executor across all processes.
of course, it requires to agree on one executor which doesn't sound realistic, but would it make sense in general?
This is essentially how an Erlang system works. There's a single executor with one "CPU" thread per CPU, and a bunch of "I/O" threads, the sum of which are shared among all Erlang applications executing under the runtime. Erlang can get away with this because it's interpreted.
But also, this is essentially what threads are... the main advantage of threads being, they coordinate raw assembly code, instead of something interpreted or JITted. OS schedulers are pretty well optimized by this point. The major overheads relate to maintaining security boundaries (e.g. swapping out the TLBs on context switch) and the cost of starting up a thread. Just having threads sitting around doesn't really cost anything.
> Erlang can get away with this because it's interpreted.
Erlang gets away with this because of its programming model; it doesn't matter if it's interpreted or JITed or if you managed to compile it ahead of time. The relevant parts of the programming model are that any function call can yield, and because it's a functional language, there's no looping without function calls, so you can't stay busy forever without yielding.
The value of green thread over native threads is the difference in numbers. I haven't tried, but I imagine most OSes would fall over if you did a native thread per connection and you had a million connections; some OSes would probably hit a thread limit before then. Of course, if you've got a million connections, you're probably starting many per second, so startup cost is important (although maybe you could do a pool if it's cheaper to clean up an otherwise dying thread than kill it and start a new one)
Edit: read the article; that was a weird rant; I'm sure there are better things to rant about on an idle windows machine (like how much disk I/O they do... especially on spinning drives with Windows 10)
Microsoft's research OS Midori supposedly did everything with async/await and got down to about one thread per processor (or at least one thread per process and processor).
In Midori all programs were written in a special dialect of C#. Safety would have been ensured by the language, but all programs it actually ran were probably also written by very competent programmers (which doesn't happen in real life ...).
If I understand things correctly, the language would have imposed an early example of "Structured Concurrency" (although that term hadn't been invented yet)
There's some fine reasons to use blocking I/O. Often you don't have anything to do until it comes back, so it's the most natural way to write it. For a very short system call, it's faster to make it blocking because you can resume the thread instantly without the overhead of however you send/handle an async reply. It avoids space leaks, because a blocking call will… block if it can't make progress, so you can't send an infinite amount of async calls.
It's ideal in the sense that you don't need to block to wait and you can avoid context switches. But obviously that means abandoning some common patterns.
"Common patterns" including, running more than one application on a system.
I've worked on several thread-per-core systems, including embedded systems in which each thread owns a core and does not make system calls or context switch. You can do that, and it's ideal performance-wise, but it does not apply to multi-process systems -- how else are they to coordinate workflows, than through kernel-visible concurrency constructs: i.e., threads?
In language implementations with built-in user-mode schedulers like Go and Erlang, you get closer to this ideal for, as they will spin up a thread per core for CPU-intensive workloads. But even there, blocking system calls are still the norm (at least with Linux, which is only very recently moving toward a generic async system call system, with io_uring), requiring many more threads per core to saturate I/O workloads. And that still doesn't address the multi-process coordination problem.
But green threading doesn't address the OP's issue of "too many threads": a process using green threads needs to spawn one thread per CPU if it plans to use the CPU's full power.
On say, a 16-core system like the OP's, you're back to 16 threads per process -- on average, more than the OP was complaining about (15).
The comments here remind me of MongoDB is web scale. You can have thousands of idle threads on a machine with one or a few CPU cores. Sure if you have several of them competing for CPU time you will lose efficiency. But at the point where the total duty cycle is less than capacity - context switch overhead you will not see any I’ll effects. I know because I have tried this. 4096 threads on a dual core $5/month VPS did beautifully for IO-bound tasks.
Also the author is examining a desktop machine. Of course it will have a bunch of idle processes. In fact I would argue that if the machine was running 100% CPU usage by a few threads all day long the author would be quite upset at the electric bill, fan noise, and the heat generated. Desktop applications are not throughput bound. They are latency bound and having a bunch of threads on standby is normal.
In the modern era, threads are more of a memory problem than a compute problem. If you spawn a thread per request (or several) it can be an issue by maintaining too much stack space. But I don't know of any reliable benchmark showing that threads or events outperform in real systems, except in cases of extremely high numbers of simultaneous contexts (i.e. very high number of ongoing requests/very low cost per request of operation).
You could just spawn threads with smaller stacks. Also afaik that memory won't ever get paged in if it isn't used? So I don't know if it matters much.
Also, idle threads aren't a problem, even on a system under load. Especially on a desktop.
Also also, there's no global threadpool, which means if you have K programs on N codes and K > N it's a moot point trying to save threads. Any desktop has K > N. So reducing thread usage in all of your processes is unlikely to help.
More to the point, only allocated pages matter from the perspective of resident memory. As long as the thread stacks are not actually too deep, it doesn't really matter very much the nominal depth.
This post has the thesis that threads are expensive in some way. Modern OS's schedulers don't do work that's O(blocked threads) to schedule. They require a small amount of memory to maintain some data structures to track it in the kernel, and some memory for a stack in userspace.
Threads as a mechanism for separating out the concerns that can run independent of each other is perfectly fine. While eventloops and thread pools are good, allowing for things like, say, a stats aggregator to not need to share a work queue with an application's work just makes for an easier-to-reason-about system.
50 comments
[ 113 ms ] story [ 212 ms ] thread7000+ threads does seem like a lot though and agree there is the general lack of caring about resource consumption in all forms, eg Electron apps. At this point having lots of threads seems almost expected by various OSs and something they solve for to reduce the cost of it so the user doesn't have to worry about it.
Isn't this the case for some older cooperative threading OSes? It's possible in theory, anyway.
Of course, there's a lot of tradeoffs between that and just keeping an idle thread around.
Seems like that is an ideal activity for a thread, and in fact why most people use them? Combining all those waits into a single thread/process means also creating a dispatcher which then either spins up a thread to do work (ahem), or blocks all other useful work. If the cost is low (and for thread it usually is), having lots of blocked threads often makes the code easier to write and maintain.
This is where Moore’s Law has brought us: if we want better performance, we need to take advantage of parallelism. Thus more threads.
Threads need to scale with CPU cores, however simply increasing #threads doesn't help. You need to split tasks of your program such that more tasks can be executed in parallel while not adding more overhead than you gain by parallelism.
Then you can distribute those tasks on as many threads as possible (while still being bound by #cores).
It's bound by the slowest shared resource in the system (Amdahl's law). That could be disk I/O, memory bandwidth, or power budget.
Agree, because spawning a new thread and killing a thread carries its own overhead so better to have the thread sleep or be paused by the OS and cpu scheduling.
Its all about getting the balance right, but I have noticed dont net apps run their gui on a single thread instead of a window(s) linked to a thread, like in a multi document interface (MDI) setup.
In an OS like Windows, on "typical" hardware, the amount of memory per thread for stacks and bookkeeping the OS and/or userland allocate is usually negligible. In this context also remember that the reserved stack sizes per thread do not refer to physical memory but virtual memory (or "committed" vs "reserved"). A typical thread usually commits 1 page for the stack and the equivalent of up to 2 pages worth for any bookkeeping[0], etc, which would put the memory committed for the 7412 threads the author mentions in the ballpark of maybe 30-100 MiB of memory actually committed.
The OS scheduler should be good enough to distinguish between runnable/waiting (sleeping) threads, eliminating the need to even look at most threads (which will be in waiting state) until some signal arrives putting them back into a runnable state.
You only really get a problem when you have too many runnable threads (fighting over the available resources) or when too many threads keep rapidly toggling between runnable and waitable, as either case would potentially lead to a lot of expensive context switches (and the former case is an indication you're trying to do too much with the physical resources available).
[0] Tho some things may commit considerably larger amounts of stack space and/or bookkeeping space. E.g. if I remember correctly java8 commits 1MiB of stack space per thread by default, while e.g. java11 merely reserves that 1MiB.
The number of threads should only become a bottleneck if you have less than 0.01% of them idle.
A single thread can then wait for events that would require some processing (IO or timers being the most common) and then it passes that unit of work to the thread pool to be processed. Windows even takes pains to ensure that the last thread to process a unit of work, might then also be used to process the next item in the queue of work to eliminate costly context switching, and invalidation of data/instruction cache.
Thread pools allow the best use of threading by: - limiting the number of threads a process creates/maintains. The thread pool itself can grow or shrink as demand changes. - limiting the number of threads that are in a runnable state by queueing work to the pool. The pool itself will be sized according to number of CPU cores, e.g. 2 * CPU cores.
The main idea behind tread pools is that any thread that's not doing useful work is a waste of OS resources, so they make sure that they are either always fed with work, or get destroyed when idle for too long. From an overall machine efficiency perspective this makes total sense.
Having hundred or even thousands of threads created and idle is massively wasteful and bad design, the OS should not be burdened with such housekeeping.
Note that Apple's GCD also works on the same principle, as does libev/libuv.
This is why thread-per-core software architectures are idiomatic for anything requiring maximum performance, especially on systems with lots of cores. Similarly, there is no requirement to spawn separate threads for task concurrency. The non-blocking coroutine-style concurrency common in high-performance server software supports much higher concurrency and with context switching costs that are much lower.
The thread-per-core coroutine-style is easier to write and maintain too -- the logic is almost entirely single-threaded. No deadlocks, races, etc to worry about because lock graph concurrency is complicated.
It wouldn't. There will only be a context switch if the thread is woken up for some reason (eg. the operation it's waiting on compeltes). If a thread is spending a majority of its time waiting, then it's not going to be doing a lot of context switches.
> This is why thread-per-core software architectures are idiomatic for anything requiring maximum performance
That's a stretch. There are some specific real-time usecases where you want to lock a specific thread to each core, but those are extremely niche and imply total control over what is running on the system.
This architecture has been idiomatic for high-performance server software for a solid decade. It only seems niche if you don't work in an area of software that cares a lot about performance. The performance difference is not small, 10x throughput on the same hardware is not uncommon.
And yes, it implies control of the resources your process nominally owns, though by no means the entire machine necessarily. Again, idiomatic, and software like high-end database engines have been designed this way for a very long time so it isn't a new idea. Designing software that takes full control of its resources, and does its own I/O and process scheduling, isn't magic, just different.
A classic multithreaded software architecture will never be remotely performance competitive with a thread-per-core architecture. It isn't expected in theory, and is never demonstrated in practice. Modern thread-per-core architecture has its origins in supercomputing, specifically to enable better scalability as the number of cores per CPU (and server) became greater than 2.
I’ve spent the last two and a half years mainly programming in single-threaded NodeJS environments with a focus on heavy same-task concurrency, and I’ve still run into deadlocks and races, despite the lack of OS threads involved.
However, your point that context switching is expensive, and is why thread-per-core architectures are so common, still holds.
Seems to not account for the fact that threads will sleep while waiting for I/O.
According to my understanding you don't need additional threads for this.
We've had this type of cpu time sharing far longer than we've had multicore CPUs.
Or perhaps you're using NodeJS-style async callbacks. Guess what happens... Your async read looks like it's just using an event queue, but behind the scenes, NodeJS calls libuv which creates a thread which blocks in read(), and when read() unblocks and returns your callback is added to the event queue.
Perhaps you're using the async/await style with the Promise API for truly async goodness. Guess what happens... Your async call still ends up calling libuv which creates a thread which blocks in read(), and your await waits until libuv's thread read() unblocks and tells the Promise that it's fulfilled.
Maybe you're using shiny io_uring() - finally (it was proposed many times before) a real async system call interface in Linux! Guess what happens... Your lovely async/await calls the pristine goodness of io_uring_enter() which... creates a kernel thread which blocks in read(), and your await waits until the kernel thread's read() unblocks and reports a result.
Even with async/await languages and frameworks, and event-driven frameworks, a blocking thread is used somewhere in the stack to read from a file.
I've measured how many threads to create for optimal throughput of small reads from NVMe storage, and the fastest number of threads varies but it's generally in the tens to low hundreds, and always more than the number of CPU cores, if the data is not already in cache.
This is a complex area to tune adaptively, and Linux itself is unfortunately not ideal for this (it doesn't tell the userspace scheduler about blocked vs preempted I/O threads). Interestingly, reducing the number of CPU cores (using taskset) while keeping the same number of threads sometimes increases I/O throughput.
(Note: The above assumes Linux, not O_DIRECT with libaio, and that data is not in cache when using io_uring).
[My journey was basically being kool-aided on various green thread variants and async and whatnot, only to realize that threads are pretty damn excellent for most use cases, and all alternatives come with subtle drawbacks.]
Do you think that io_uring (or similar) has a ubiquitous future (as opposed to niche – which there is clearly a demand for atm)? Or is combining kernel and user space scheduling is a forever-hard-to-tune problem? Also, do you see any potential improvements to traditional threading/scheduling that could be helpful alleviating some of the resource hogging issues (which are real but over-hyped imo)? Thanks.
... no. Not if you rely on blocking system calls. Or wait for anything that isn't the CPU. Or run more than one application per CPU on your system. Or manage concurrent code flows at any level higher than source code.
But now you have more than one application on the machine? Or even, code from multiple vendors which all need to issue I/O concurrently? How do you integrate their work with your custom scheduler?
You need some centralized, system-wide scheduler everyone agrees on.
That's what threads are.
Currently every app that uses tasks has it's own executor with a corresponding thread pool, which may be more efficiently used if it was shared by using a single async executor across all processes.
of course, it requires to agree on one executor which doesn't sound realistic, but would it make sense in general?
But also, this is essentially what threads are... the main advantage of threads being, they coordinate raw assembly code, instead of something interpreted or JITted. OS schedulers are pretty well optimized by this point. The major overheads relate to maintaining security boundaries (e.g. swapping out the TLBs on context switch) and the cost of starting up a thread. Just having threads sitting around doesn't really cost anything.
Erlang gets away with this because of its programming model; it doesn't matter if it's interpreted or JITed or if you managed to compile it ahead of time. The relevant parts of the programming model are that any function call can yield, and because it's a functional language, there's no looping without function calls, so you can't stay busy forever without yielding.
The value of green thread over native threads is the difference in numbers. I haven't tried, but I imagine most OSes would fall over if you did a native thread per connection and you had a million connections; some OSes would probably hit a thread limit before then. Of course, if you've got a million connections, you're probably starting many per second, so startup cost is important (although maybe you could do a pool if it's cheaper to clean up an otherwise dying thread than kill it and start a new one)
Edit: read the article; that was a weird rant; I'm sure there are better things to rant about on an idle windows machine (like how much disk I/O they do... especially on spinning drives with Windows 10)
In Midori all programs were written in a special dialect of C#. Safety would have been ensured by the language, but all programs it actually ran were probably also written by very competent programmers (which doesn't happen in real life ...).
If I understand things correctly, the language would have imposed an early example of "Structured Concurrency" (although that term hadn't been invented yet)
<http://joeduffyblog.com/2015/11/19/asynchronous-everything/>
And it means backtraces in gdb work.
I've worked on several thread-per-core systems, including embedded systems in which each thread owns a core and does not make system calls or context switch. You can do that, and it's ideal performance-wise, but it does not apply to multi-process systems -- how else are they to coordinate workflows, than through kernel-visible concurrency constructs: i.e., threads?
In language implementations with built-in user-mode schedulers like Go and Erlang, you get closer to this ideal for, as they will spin up a thread per core for CPU-intensive workloads. But even there, blocking system calls are still the norm (at least with Linux, which is only very recently moving toward a generic async system call system, with io_uring), requiring many more threads per core to saturate I/O workloads. And that still doesn't address the multi-process coordination problem.
But green threading doesn't address the OP's issue of "too many threads": a process using green threads needs to spawn one thread per CPU if it plans to use the CPU's full power.
On say, a 16-core system like the OP's, you're back to 16 threads per process -- on average, more than the OP was complaining about (15).
What does he think is the point of green threads? Fibres?
Why does he think some languages only allow one thread to run at a time but still have threads?
Answer to all these questions is: he’s mistaken about the original purpose of threads.
Also the author is examining a desktop machine. Of course it will have a bunch of idle processes. In fact I would argue that if the machine was running 100% CPU usage by a few threads all day long the author would be quite upset at the electric bill, fan noise, and the heat generated. Desktop applications are not throughput bound. They are latency bound and having a bunch of threads on standby is normal.
Also, idle threads aren't a problem, even on a system under load. Especially on a desktop.
Also also, there's no global threadpool, which means if you have K programs on N codes and K > N it's a moot point trying to save threads. Any desktop has K > N. So reducing thread usage in all of your processes is unlikely to help.
Threads as a mechanism for separating out the concerns that can run independent of each other is perfectly fine. While eventloops and thread pools are good, allowing for things like, say, a stats aggregator to not need to share a work queue with an application's work just makes for an easier-to-reason-about system.
I wonder why zero is only "almost" impossible. Don't you always need one thread to be called a process?