I was under the impression the `SO_REUSEPORT` implementation on Linux was rather broken, as evidenced by HAProxy's graceful reload still dropping packets[1]. I'm curious as to why Nginx has implemented this when you can design your Nginx config around the problem while not encountering the broken implementation.
The issue that HAProxy is encountering with SO_REUSEPORT is not an issue for Nginx's use case.
In the HAProxy case one of the sockets belongs to the old server which is going away, and a new connection can be assigned to that socket right before it does so. For Nginx, all of the sockets are accepting new connections.
What happens during the reload case for nginx? At reload time, the previous-generation worker that bound to the socket with SO_REUSEPORT will need to close its listener, just like haproxy.
When reloading, nginx starts up a new set of workers and duplicates the sockets across to them with SCM_RIGHTS. None of the sockets is closed - only duplicate file descriptors.
I secretly hope Nginx could implement socket fd sharing via sendmsg, so my websocket worker have direct access to the client TCP connection. No more relaying.
13 comments
[ 2.7 ms ] story [ 37.7 ms ] thread[1] http://engineeringblog.yelp.com/2015/04/true-zero-downtime-h...
And if it's offloading some work onto the kernel, having a popular tool support it may push Linux developers to fix the support.
In the HAProxy case one of the sockets belongs to the old server which is going away, and a new connection can be assigned to that socket right before it does so. For Nginx, all of the sockets are accepting new connections.