Rack 3 has better support for web servers like Falcon[0] that are designed to take advantage of async or blocking vs non-blocking IO throughout the stack.
Falcon looks cool, and it seems it could facilitates a dreamed "rails deploy" in the future, but substituting C (Nginx) for Ruby should have some type of big performance penalty.
Regarding wait_for_less_busy_worker on the surface it seems suboptimal to add a wait time before responding. Can someone explain why this is the best solution?
I may have this wrong, but here's my best understanding of it:
Ruby supports multi-threading, but unless you're using the new (and experimental) Ractor feature, you're subject to the global interpreter lock in most cases (with a few important and useful exceptions, like some kinds of I/O). That means that Ruby servers will typically employ multi-processing in addition to (or in place of) multi-threading as a way to increase performance and use multiple CPUs - otherwise, multiple threads just end up competing for the global interpreter lock and the additional threads don't increase performance as much as you would hope, especially if serving those requests requires any actual work to be done in Ruby code.
Puma supports a multi-processing mode, where a main Puma process forks multiple workers (each running multiple threads), and each worker listens on the same socket. The linux kernel distributes the load between the workers, and then the workers distribute the load internally between their threads. Since the global interpreter lock is a per-process thing, this is a pretty effective way to get more throughput for a Ruby server.
The problem is that you can't directly control how the kernel is going to balance incoming requests across the multiple workers listening on a socket. Because Ruby does support some instances where threads can run concurrently - like network I/O - it's possible that the kernel may end up handing off multiple requests to one worker process when there were others that were idle and could have handled the request. Doesn't sound like a big deal - but because most threaded Ruby operations do not run concurrently that means that the actual Ruby code that needs to process that request is going to be competing for the global interpreter lock.
So basically this allows a worker process that is already handling requests to insert a tiny delay before accepting another one - which gives an idle worker process a chance to accept it instead. On balance, this means that you'll get higher utilization of the CPU resources available to you and will often result in a lower average latency for all requests.
We tried to throw Puma at Rails a few times. Would not recommend. If you're trying to add threading to a Ruby I would question how safe you really think dynamic languages are. Fortunately Ruby/Puma/Rails have been out of favor for a while, but if you're stuck with them, I'd just accept your 2-4x higher AWS bill for the hardware you need for Rails rather than add dangerous threading abstractions.
I have no clue what Puma is in this context, but I really think people should do a quick search before they pick names, particularly with a number, and especially when a name and a number have a horrible history:
20 comments
[ 3.1 ms ] story [ 35.4 ms ] threadNames are hard.
Intel's Puma processors were first released in 2012. Ruby's Puma is from 2011.
Maybe the same for Puma?
[0]: https://github.com/socketry/falcon
Looking forward, though.
Ruby supports multi-threading, but unless you're using the new (and experimental) Ractor feature, you're subject to the global interpreter lock in most cases (with a few important and useful exceptions, like some kinds of I/O). That means that Ruby servers will typically employ multi-processing in addition to (or in place of) multi-threading as a way to increase performance and use multiple CPUs - otherwise, multiple threads just end up competing for the global interpreter lock and the additional threads don't increase performance as much as you would hope, especially if serving those requests requires any actual work to be done in Ruby code.
Puma supports a multi-processing mode, where a main Puma process forks multiple workers (each running multiple threads), and each worker listens on the same socket. The linux kernel distributes the load between the workers, and then the workers distribute the load internally between their threads. Since the global interpreter lock is a per-process thing, this is a pretty effective way to get more throughput for a Ruby server.
The problem is that you can't directly control how the kernel is going to balance incoming requests across the multiple workers listening on a socket. Because Ruby does support some instances where threads can run concurrently - like network I/O - it's possible that the kernel may end up handing off multiple requests to one worker process when there were others that were idle and could have handled the request. Doesn't sound like a big deal - but because most threaded Ruby operations do not run concurrently that means that the actual Ruby code that needs to process that request is going to be competing for the global interpreter lock.
So basically this allows a worker process that is already handling requests to insert a tiny delay before accepting another one - which gives an idle worker process a chance to accept it instead. On balance, this means that you'll get higher utilization of the CPU resources available to you and will often result in a lower average latency for all requests.
The PR that added this in Puma 5.0 is here: https://github.com/puma/puma/pull/2079
https://www.theregister.com/2017/04/11/intel_puma_6_arris/
It also wouldn't hurt if they main page for a project had just a wee bit more information about what it actually is and what it actually does.
To be fair Puma has been out and named for over 10 years. It's one of the most popular app servers for Ruby.
The HN post links to the 6.0 upgrade guide, the repo's readme file goes into more detail on what it is on the first line:
Puma is a simple, fast, multi-threaded, and highly parallel HTTP 1.1 server for Ruby/Rack applications.