Jul 18·6 min· PV

IO Multiplexing

A brief walkthrough of how select, poll, and epoll are implemented.

Tip

This note focuses on network I/O. For an introduction to the 5 I/O models, see io_models

Select

select is an I/O multiplexing mechanism used to monitor multiple file descriptors at once for I/O readiness events, such as readable, writable, or exceptional conditions.

int select(int nfds, fd_set *readfds, fd_set *writefds,           fd_set *exceptfds, struct timeval *timeout);

For the readiness conditions, see UNIX Network Programming Volume 1, section 6.3.1. The figure below summarizes them:

timeout

The timeout parameter sets the wait time, in one of three ways:

No waiting: poll the file descriptors and return immediately, with the time in timeval set to 0
Wait until a file descriptor is ready: specify a wait time in timeval; select blocks until a file descriptor becomes ready or the timeout expires
Wait forever: set timeval to NULL; select blocks until some file descriptor is ready

struct timeval

struct timeval {    long tv_sec;  // seconds    long tv_usec; // microseconds};

Fd Set

fd_set is typically a bitmap, implemented internally as an array of long integers. Each bit in the array corresponds to a file descriptor.

The size of this bitmap is fixed, determined by the constant FD_SETSIZE (usually 1024 on Linux). This is hard-coded in the Linux kernel; changing the fd_set size requires modifying the source and recompiling glibc and the kernel.

When writing C code, you can use the following macros to manipulate an fd_set:

FD_ZERO(fd_set *fdset);          // clear all bits in fdsetFD_SET(int fd, fd_set *fdset);   // turn on the bit for fd in fdsetFD_CLR(int fd, fd_set *fdset);   // turn off the bit for fdFD_ISSET(int fd, fd_set *fdset); // is the bit for fd on?

Example

For example, to monitor file descriptors 3, 5, and 8 for readability, set up the fd_set like this:

fd_set read_fds;FD_ZERO(&read_fds);FD_SET(3, &read_fds); // watch fd 3FD_SET(5, &read_fds); // watch fd 5FD_SET(8, &read_fds); // watch fd 8select(9, &read_fds, NULL, NULL, &timeout);

When select finishes, it returns the total number of ready file descriptors and modifies the fd_set accordingly, setting the bits of the ready file descriptors to 1. Because of this, you usually need to re-populate the fd_set before each new call to select.

nfds

nfds (number of file descriptors) is usually set to the highest file descriptor in the passed-in sets plus 1, that is:

UNIX Network Programming Volume 1 notes that “the reason this argument exists, along with the burden of calculating its value, is purely for efficiency.” To understand why the parameter matters, first imagine there were no nfds parameter:

select would copy the entire 1024-bit bitmap (fd_set) into kernel space, then copy the bitmap back to user space when the call finishes. That overhead is not trivial.

Yet an ordinary process typically checks far fewer than 1024 file descriptors. Taking the code above as an example, we only care about descriptors 3, 5, and 8; the remaining 1015 bits are irrelevant, and transferring them is unnecessary.

With the nfds parameter, select does not have to copy the full 1024-bit bitmap. Instead, it uses nfds to decide up to which byte to copy, saving CPU time and memory bandwidth.

Poll

poll offers functionality similar to select. The kernel still scans the array linearly, so the time complexity remains .

The differences are that the size of the fds array you pass in can grow as needed (still bounded by system resources), and the API is a bit cleaner.

Warning

On Linux, the number of file descriptors each process can open is limited, typically defaulting to 1024. You can view and change the value with the ulimit command.

select’s file descriptor limit is additionally constrained by a hard-coded implementation constant (usually FD_SETSIZE, default 1024). Even if the system allows more fds, select still cannot support them.

# Maximum number of file descriptors the current process can openulimit -n# System-wide limit on total open files (across all processes)cat /proc/sys/fs/file-max

int poll(struct pollfd *fds, unsigned long nfds, int timeout);struct pollfd {    int fd;         // file descriptor    short events;   // events of interest on fd    short revents;  // events that occurred on fd};

Each bit of events represents one kind of event; like select, they fall into three categories: read, write, and exception. Run man poll in a shell for the detailed event types.

revents is filled in by the kernel after the poll call, indicating the events that actually occurred.

Epoll

epoll is a Linux-specific I/O multiplexing mechanism designed for handling large numbers of file descriptors. It is more efficient than select and poll, especially with many concurrent connections. For a deep dive into epoll, see 图解 | 深入揭秘 epoll 是如何实现 IO 多路复用的！

Feature	select	poll	epoll
Interface standard	POSIX	POSIX	Linux-specific
fd count limit	Limited (usually 1024)	Unlimited	Unlimited
fd representation	Bitmap	Array	Kernel structure
Copies fd set	On every call	On every call	Registered once
Scan complexity	O(n)	O(n)	O(1)~O(active fds)

Epoll LT & ET

Dimension	Level-triggered (LT)	Edge-triggered (ET)
Trigger condition	Fires continuously as long as the state holds (e.g. buffer non-empty)	Fires once, only on the “not ready → ready” transition
Notification frequency	High (keeps notifying until data is drained)	Low (only on state changes)
Data handling requirement	Can process in multiple passes, no need to finish at once	Must drain all current data in one go
Requires non-blocking I/O	Optional (works with blocking I/O too)	Required (otherwise threads may block)
Programming complexity	Low (hard to get wrong)	High (must handle edge cases to avoid data loss)
Performance	Moderate	Higher (less notification overhead)

Ref

你管这破玩意叫 IO 多路复用？
《UNIX Network Programming》- W. Richard Stevens