Virtual Memory¶

The idea¶

Every memory address a program uses — every pointer value — is a virtual address. It is not a location in RAM. On every load and store, a hardware unit in the CPU, the MMU (memory management unit), translates the virtual address into a physical address using tables that the operating system maintains for each process. The program never sees physical addresses.

This single level of indirection is what makes possible almost everything a modern OS does with memory.

What the indirection buys¶

Pages: the unit of translation¶

Translation is done per page — a fixed-size, aligned block — not per byte. A virtual address splits into a virtual page number (which page) and an offset (where inside the page). The offset passes through unchanged; only the page number is translated to a physical frame number.

Translation step by step (x86-64, 4 KB pages)¶

A 64-bit pointer on x86-64 uses only the low 48 bits (57 with 5-level paging on recent CPUs); the upper bits must be copies of bit 47 ("canonical" addresses). The 48 bits are split into four 9-bit table indexes and a 12-bit offset:

virtual address 0x00007f3e1c0040a8

bits:  47..39     38..30     29..21     20..12      11..0
       PML4 idx   PDPT idx   PD idx     PT idx      offset
       0x0fe      0x0f8      0x0e0      0x004       0x0a8

Each table is exactly one 4 KB page holding 512 eight-byte entries (2⁹ = 512, which is why each index is 9 bits). The CR3 register points at the current process's top-level table; switching processes means loading a different CR3. Each level's entry gives the physical address of the next table; the last level's entry gives the frame.

Why multi-level? A flat table for a 48-bit space with 4 KB pages would need 2³⁶ entries × 8 bytes = 512 GB per process. With a tree, tables exist only for regions actually mapped: a small process needs a handful of 4 KB tables. The cost is that one translation needs four memory reads — which is why the TLB exists.

Page table entry bits¶

The TLB¶

The translation lookaside buffer caches recent page-number → frame translations. A TLB hit costs nothing extra; a miss triggers a page walk of up to four dependent memory reads (partly cached in the CPU's data caches and dedicated page-walk caches).

Typical sizes: an L1 data TLB of ~64 entries and a shared L2 TLB of ~1,500–3,000 entries. With 4 KB pages, 2,048 entries cover only 8 MB. A database touching a 50 GB buffer pool at random misses the TLB on nearly every access. Mitigations:

• Huge pages: one 2 MB entry covers 512 small pages; 2,048 entries then cover 4 GB. Databases (PostgreSQL huge_pages, Oracle, JVM heaps) use explicitly reserved huge pages for large buffer pools. See Paging.
• Larger base pages: Apple Silicon's 16 KB pages quadruple TLB reach compared to 4 KB.
• Locality: keep hot data dense so fewer pages are touched.

Context switches and the TLB¶

TLB entries belong to one address space. Historically, switching processes flushed the TLB, so the next process started with every access missing. Modern CPUs tag entries with an address space ID (PCID on x86, ASID on ARM), so entries of several processes coexist and a switch flushes nothing. Threads of one process share an address space — switching between them never needed a flush, which is part of why thread switches are cheaper than process switches (see Process vs. Thread).

When the kernel removes or downgrades a mapping (munmap, mprotect, page reclaim, copy-on-write), every CPU that may have cached it must invalidate its TLB entry. The kernel sends inter-processor interrupts: a TLB shootdown. In a many-threaded process that maps and unmaps memory frequently, shootdowns become a real cost — one of the arguments against mmap-based database I/O (see Linux I/O Stack and Page Cache).

Kinds of mappings¶

Every region in the address space is created by the kernel's mmap machinery, with two independent choices:

import "golang.org/x/sys/unix"

// File-backed, shared, read-only: read an SSTable through memory.
f, _ := os.Open("000123.sst")
st, _ := f.Stat()
table, _ := unix.Mmap(int(f.Fd()), 0, int(st.Size()), unix.PROT_READ, unix.MAP_SHARED)

// Anonymous, private: a 1 GiB region, physical memory supplied lazily.
region, _ := unix.Mmap(-1, 0, 1<<30, unix.PROT_READ|unix.PROT_WRITE, unix.MAP_ANON|unix.MAP_PRIVATE)

Both return a []byte that is not Go heap memory: the garbage collector ignores it, and it stays valid until unix.Munmap. The Go runtime itself gets its heap arenas from anonymous private mappings like the second one.

The physical pages of a file-backed mapping are the page cache pages for that file — mmap and read see the same memory, which is why mapping a file does not double its memory use.

Overcommit and the OOM killer¶

Because reserving address space is free, Linux by default lets processes reserve more memory than RAM + swap. The policy is vm.overcommit_memory:

Worked example — overcommit. A process mmaps a 10 GB region but only ever touches 100 MB. The call succeeds; VSZ shows ~10 GB, RSS ~100 MB. Measured on a Mac: mapping 1 GiB caused zero page faults and no change in resident memory; touching 100 MiB then caused one fault per page and RSS grew by exactly 100 MB.

The flip side: if processes actually touch more than physical memory can hold (after reclaiming page cache and swapping), the kernel must take memory back by force. The OOM killer picks a victim by a score based mainly on its memory size (adjustable per process via /proc/<pid>/oom_score_adj; -1000 exempts a process) and kills it with SIGKILL. In containers, the same happens per cgroup when the memory limit is reached — the "OOMKilled" status in Kubernetes. For a database, this means memory budgets must include caches, memtables, per-connection buffers, and allocator fragmentation, not just "data size."

Kernel space¶

The upper part of every process's address space maps the kernel, marked supervisor-only, so a system call can enter the kernel without switching page tables. After the Meltdown CPU vulnerability (2018), which let user code speculatively read kernel memory, kernels adopted KPTI (kernel page-table isolation): user mode runs with page tables that map almost none of the kernel, and every system call and interrupt switches tables. PCID keeps the TLB cost tolerable, but system-call-heavy workloads lost several percent of throughput — a reminder that the cost of a system call is partly a cost of virtual memory.

Worked example — isolation. Two unrelated processes both use virtual address 0x00007f0000000000 in their heaps. There is no conflict: each process's page tables translate that address to a different physical frame, and the kernel loads the right tables (CR3) on every context switch. Process A writing to that address can never touch B's data, because A's tables simply contain no entry leading to B's frames.

Common pitfall¶

Assuming that memory "allocated" is memory "used," or the reverse. malloc and mmap succeed by reserving virtual address space, so a service can start with a 64 GB cache configured on a 32 GB machine and run fine for days — until the cache actually fills, the pages are touched, and the OOM killer ends the process at the worst moment. The reverse error is alarming at a large VSZ: thread stacks, reserved arenas, and mapped-but-cold files inflate virtual size without using RAM. Size memory budgets by what will be touched at steady state, and watch RSS (and per-cgroup memory usage in containers) against the limit, not VSZ.