System Calls¶

What a system call is¶

A system call is how a user-space program asks the kernel to do something it is not allowed to do itself: read a file, send a packet, allocate address space, create a process or thread, wait on a lock. Every interaction a program has with the outside world goes through one. A storage engine's performance and correctness depend directly on which system calls it makes, how many, and how it handles their failures.

Why programs can't just do it themselves¶

The CPU runs in different privilege levels: user mode (x86 ring 3, ARM EL0) and kernel mode (ring 0, EL1). In user mode, instructions that touch hardware (I/O ports, page tables, interrupt control) fault, and memory pages marked supervisor-only are inaccessible. This is what stops a buggy or malicious process from reading another process's memory, overwriting the disk directly, or crashing the machine. The kernel is the only code with that power, and a system call is the one controlled door into it.

What happens during a system call¶

1. The libc wrapper puts the system call number (e.g. read is 0 on x86-64 Linux) and up to six arguments into registers.
2. The syscall instruction (svc on ARM) switches to kernel mode and jumps to the kernel's entry point. The kernel switches to the thread's kernel stack — user-space memory is never trusted for this.
3. The kernel looks up the handler in its system call table, validates every argument (is fd open in this process? is buf a user address the process may write?), and does the work.
4. Data crosses the boundary by explicit copies (copy_from_user, copy_to_user) that safely handle faults on bad pointers — this is why passing an invalid buffer returns EFAULT instead of crashing the kernel.
5. sysret returns to user mode. On failure, the raw kernel return is -errno; the libc wrapper converts it to -1 and sets the thread-local errno.

What it costs¶

Measured¶

From Go on an Apple Silicon Mac (Go 1.26.4), syscall.Getppid() — a system call that does essentially nothing — costs about 145 ns per call. That is the fixed price of the round trip: mode switch, entry/exit code, register saving, security mitigations, plus the Go runtime telling the scheduler the goroutine is in a syscall. For comparison, a Go function call that is not inlined costs about 2 ns.

The fixed cost dominates small operations. Reading a 64 MiB file that is already in the page cache through (*os.File).Read with different buffer sizes (the 1- and 16-byte runs read only the first 8 MiB):

Up to a few hundred bytes, the time per call stays near ~470 ns regardless of size: almost all of it is overhead, so throughput is proportional to buffer size. From 4 KB upward, the copy itself starts to dominate, and beyond ~64 KB throughput approaches memory-copy speed. The gap between 1-byte and 1 MB reads is ~9,000x for the same data.

Worked example. Copying a 64 KB file one byte at a time means 65,536 read() calls; at ~470 ns each, ~30 ms spent crossing the user/kernel boundary before counting any actual I/O. One 64 KB read() pays the fixed cost once (~4 µs). This is the reason buffered I/O exists (bufio.Reader and bufio.Writer in Go, stdio in C): fill a user-space buffer with one large read and serve small reads from it. The same effect on a socket — 5.8 µs per unbuffered 16-byte Write against 8 ns with bufio — is measured in Network Programming in Go.

Where the time goes¶

The last row is the important one: a system call that blocks is expensive not because of the mode switch but because of the scheduler round trip.

System calls that aren't: the vDSO¶

Some frequently used calls only read kernel-maintained data. Linux maps a small shared library, the vDSO, into every process, letting clock_gettime, gettimeofday, and getcpu run entirely in user mode by reading a kernel-updated memory page. That is why Go's time.Now() costs tens of nanoseconds rather than a full system call (the Go runtime calls the vDSO directly on Linux) — and why timestamping every operation in a database is affordable.

Reducing the number of system calls¶

Real-world usage: Go's runtime batches I/O through its netpoller — net.Conn.Read() does not block a thread in read(); goroutines waiting on sockets are parked, and a single epoll_wait (or kqueue) call reports which of thousands of connections are ready.

Handling errors correctly¶

At the C level, system call wrappers return -1 and set errno; in Go, golang.org/x/sys/unix returns the same codes as syscall.Errno error values. Either way, robust code must handle several outcomes that are not failures of the operation:

At the raw system-call level (golang.org/x/sys/unix), the caller handles both cases itself:

func writeFully(fd int, data []byte) error {
    for len(data) > 0 {
        n, err := unix.Write(fd, data)
        if err == unix.EINTR {
            continue // interrupted before writing anything: retry
        }
        if err != nil {
            return fmt.Errorf("write: %w", err) // real error (ENOSPC, EIO, ...)
        }
        data = data[n:] // short write: continue with the rest
    }
    return nil
}

The os and net packages already do this: (*os.File).Write retries on EINTR and loops over short writes, returning an error only if it could not write everything, and io.ReadFull loops over short reads. Code that calls unix.Read/unix.Write directly — or any io.Reader contract (a single Read may return fewer bytes than asked) — must handle short counts itself. Go returns the error value from the call, so the C pitfall of reading a global errno after some other call has overwritten it does not exist.

Observing system calls¶

strace intercepts every call via ptrace and can slow a program by 10x or more; use it to understand behavior, not to measure performance. The summary mode is often the fastest way to discover that a "slow database" is making a million 100-byte writes or an fsync per operation.

Restricting system calls¶

Because the system call interface is the kernel's entire attack surface from user space, sandboxes restrict it: seccomp filters on Linux allow only a whitelist of calls (Docker's default profile blocks dozens; Chrome's renderer allows very few). A storage engine running under such a profile must not depend on unusual calls — for example, io_uring is blocked by default in some container runtimes for exactly this reason.

Common pitfall¶

Assuming a function from the standard library is "just a function." A log.Printf to an unbuffered os.Stderr on every request, os.Stat in a loop to check whether a file exists, writing to an *os.File directly in small pieces instead of through a bufio.Writer, opening and closing the same file per request — each hides one or more system calls per invocation, and at a million operations per second the ~0.1–0.5 µs fixed cost of each becomes most of the CPU budget. Use strace -c (or perf trace -s) on a running benchmark: the count of calls per operation is usually more revealing than any CPU profile.