Async I/O and io_uring¶

The problem: one blocking read at a time wastes the device¶

Storage Hardware showed that an NVMe SSD reaches its rated IOPS only with many requests in flight. A blocking pread puts the calling thread to sleep until the I/O completes, so one thread can keep exactly one request in flight.

Worked example: a drive with 80 µs read latency, driven by one thread doing blocking 4 KB reads, completes at most 1 / 80 µs = 12,500 IOPS — about 2% of a drive rated for 600K IOPS. To use the whole device, you need roughly 600K × 80 µs = 48 requests in flight at all times (Little's law: concurrency = throughput × latency).

There are three ways to get there.

Option 1: a thread pool of blocking calls¶

N threads each blocking on pread gives N requests in flight. This is simple, portable, works with buffered I/O, and is what LevelDB and (by default) RocksDB do: foreground reads happen on the user's threads, compactions on a background pool. See Thread Pools.

The cost: each thread needs a stack (often 8 MB of virtual memory), and every I/O costs two context switches (sleep and wake). At hundreds of thousands of IOPS, the context-switch overhead becomes a measurable share of CPU.

Option 2: Linux AIO (io_submit) — and why it is mostly legacy¶

Linux's original asynchronous interface, libaio, lets one thread submit many reads and collect completions later. It only works properly with O_DIRECT (on buffered files it silently becomes synchronous), and each submission and completion still costs a system call. It was used by ScyllaDB and MySQL, and has been largely superseded by io_uring.

Option 3: io_uring¶

io_uring (Linux 5.1+, 2019) is built around two ring buffers shared between user space and the kernel:

1. The application writes submission queue entries (SQEs) — "read 4 KB from fd 7 at offset X into buffer B" — directly into shared memory.
2. One io_uring_enter() system call submits all pending entries at once (with SQPOLL mode, a kernel thread polls the queue and even this syscall disappears).
3. The kernel posts completion queue entries (CQEs) with each result.
4. The application reads completions from shared memory without a system call.

So one thread can keep dozens of reads in flight with a handful of system calls in total. io_uring works for buffered and direct I/O and supports reads, writes, fsync, openat, network sockets, and more.

io_uring and Go¶

Go's standard library does not use io_uring: file I/O goes through blocking pread/pwrite calls, each holding an OS thread while it runs (Runtime Scheduler). Third-party packages (for example github.com/pawelgaczynski/giouring and github.com/iceber/iouring-go) expose the rings directly; they are used by Go storage and network projects that need the last step of efficiency, and require Linux 5.x+.

What every Go program can do today is keep many reads in flight with goroutines — the runtime turns them into concurrent blocking calls on separate threads:

// readBlocks reads every offset with at most `depth` reads in flight.
func readBlocks(f *os.File, offsets []int64, depth int) error {
    sem := make(chan struct{}, depth)
    var g errgroup.Group
    for _, off := range offsets {
        sem <- struct{}{}
        g.Go(func() error {
            defer func() { <-sem }()
            buf := make([]byte, 4096)
            _, err := f.ReadAt(buf, off) // pread: safe for concurrent use
            return err
        })
    }
    return g.Wait()
}

Measured on the Apple M1 Pro's internal NVMe SSD (Go 1.26.4): 4,000 random 4 KB reads from a 4 GB file, with the page cache bypassed (F_NOCACHE on macOS, the role O_DIRECT plays on Linux):

Up to queue depth 4 the device absorbed the extra parallelism with no added latency: 4x the throughput for free. Beyond that, throughput kept rising but each read waited in the device queue, and latency grew 5x by depth 64 — the familiar queueing curve (Load Testing and Tail Latency). The last column is the cost io_uring removes: to keep 16 or 64 blocking reads in flight, the Go runtime needed extra OS threads. At thousands of concurrent reads that becomes thousands of threads, which is why high-IOPS engines move to io_uring or to a bounded pool of I/O goroutines sized for the device.

RocksDB uses io_uring for MultiGet (many independent point lookups batched together) and for asynchronous prefetching during scans. ScyllaDB's Seastar framework and TigerBeetle are built around asynchronous I/O from the ground up.

Choosing between the options¶

The thread-pool model is usually enough because each storage operation spends far more time in CPU work (decompression, checksums, merging) than the context-switch overhead of a blocking call. io_uring pays off where the bottleneck is genuinely device queue depth.

Asynchronous design changes the whole program¶

The deeper cost of async I/O is architectural. A blocking call keeps its state on the thread's stack; an asynchronous request needs its state stored somewhere until the completion arrives, so code becomes callbacks, state machines, or coroutines. Go sidesteps this for sockets: goroutines keep their state on their own stacks while the netpoller does the asynchronous part underneath (Network Programming in Go). The event loop model behind Node.js and Nginx is the same idea applied to sockets. Thread-per-core designs (Seastar) go further: one thread per CPU core, no shared data between cores, and all I/O asynchronous.

Common pitfall¶

Switching to io_uring and expecting a speedup at queue depth 1. If the application still submits one read and immediately waits for it, io_uring does exactly what pread did, with extra bookkeeping. The gain comes only from having many I/Os in flight — so the work has to be restructured to discover multiple independent reads up front (batched lookups, prefetching the next blocks of a scan) before the interface can help.