While epoll() is a clever interface for handling hundreds of thousands of ongoing connections, it seems badly designed for efficiently coping with hundreds of thousands of short-lived connections.
Every time a new connection is established, you need to make a syscall to add that new FD to the epoll set. Each syscall can only register one FD at a time. If your server is accepting thousands of connections a second, it's very likely that you are going to accept() many new connections on each event loop around epoll_wait(). This leads to lots and lots of syscalls, which although quite fast on Linux, is still not going to be greatly efficient.
Compare to FreeBSD's kqueue/kevent architecture: It's broadly similar to epoll, but you only need to make one syscall per loop: you call kevent() with a changelist of FDs, and it will return a list of all new events across the new + existing FDs in the set.
As of earlier this year, that should be mitigated with the addition of IORING_OP_EPOLL_CTL, which lets you do the equivalent of epoll_ctl() from io_uring. So you can put many EPOLL_CTL_ADD commands in the ring without incurring syscall overhead for each one.
I would be surprised if it sees a lot of use. As soon as people start building eventloops around io_uring, they might not care at all anymore about epoll.
3 comments
[ 5.0 ms ] story [ 16.8 ms ] threadEvery time a new connection is established, you need to make a syscall to add that new FD to the epoll set. Each syscall can only register one FD at a time. If your server is accepting thousands of connections a second, it's very likely that you are going to accept() many new connections on each event loop around epoll_wait(). This leads to lots and lots of syscalls, which although quite fast on Linux, is still not going to be greatly efficient.
Compare to FreeBSD's kqueue/kevent architecture: It's broadly similar to epoll, but you only need to make one syscall per loop: you call kevent() with a changelist of FDs, and it will return a list of all new events across the new + existing FDs in the set.