I/O Models and Event Loop

Blocking I/O: read() blocks until data is available. Simplest model — one thread per connection, each thread sleeps while waiting for I/O. Problem: each blocked thread consumes ~1MB of stack. 10,000 connections = 10GB of stack. Context switch overhead for 10,000 threads = significant.

graph TD subgraph block["Blocking: 1 thread per connection"] b1["10,000 connections"] --> b2["10,000 threads<br/>most sleeping, ~10GB stack RAM"] end subgraph ep["epoll: few threads, many fds"] e1["10,000 connections"] --> e2["epoll_wait() returns<br/>only the READY fds"] e2 --> e3["1 thread processes<br/>whichever fds are ready"] end

Non-blocking I/O: read() with O_NONBLOCK — returns EAGAIN immediately if no data. Application must poll manually. Wastes CPU checking repeatedly. Correct but inefficient without multiplexing.

I/O multiplexing — select/poll: Wait for any of a set of fds to become ready. select() limited to 1024 fds. poll() removes the limit but both are O(n): kernel scans all fds on every call, regardless of how many are ready.

epoll (Linux): epoll_create creates a kernel interest list. epoll_ctl(ADD, fd) registers a callback on the fd's wait queue (O(1) per registration). epoll_wait blocks and returns only the ready fds — O(1) regardless of total registered fds. The kernel uses internal callbacks rather than scanning. This is why Node.js, nginx, and Go's netpoller handle 100K+ connections efficiently.

Level-triggered (LT, default): notifies as long as data is available. Edge-triggered (ET): notifies only on transition from not-ready to ready. ET requires draining the fd completely on each event — missed data if you read partially.

Why async I/O scales better: In blocking I/O, N concurrent connections require N threads. Each blocked thread wastes ~1MB RAM and a kernel thread stack. Thread context switches at 10K connections = significant scheduling overhead. In async I/O (epoll-based), one or a few threads handle all connections. A thread only executes when there is actual work to do — it is never blocked idle. CPU time is spent on computation, not on sleeping threads.

Concrete comparison for 10,000 concurrent connections each doing 1 request/second: - Blocking (one thread per connection): 10,000 threads, most sleeping. ~10GB RAM for stacks. ~10,000 context switches/second. - Async (epoll, one thread): 1 thread, always active. ~1MB RAM. ~10,000 epoll events/second, no unnecessary context switches.

Go's netpoller: All network I/O is non-blocking internally. net.Read() parks the goroutine (removes it from the run queue, registers the fd with epoll). The OS thread continues running other goroutines. When epoll reports data ready, the goroutine is unparked. Goroutines appear to use blocking I/O (simple sequential code) but never block an OS thread.

Common pitfall

Using edge-triggered (ET) epoll and reading only some of the available data per event is a silent data-loss bug waiting to happen — ET fires once on the not-ready-to-ready transition and then goes quiet; if the handler doesn't loop read() until it gets EAGAIN (fully draining the buffer), any data that arrived beyond what was read in that one pass never triggers another event, and just sits in the kernel buffer unnoticed. Level-triggered (LT) forgives a partial read (it fires again as long as data remains), which is why LT is the safer default unless the code is specifically written to drain fully on every ET event.