Copy-on-Write¶

The idea¶

Copy-on-write (CoW) postpones copying data until someone actually modifies it. Two owners share one copy, marked read-only; the first write by either owner triggers a private copy of just the part being written. If neither writes, nothing is ever copied.

The idea appears at every layer of a system:

This page starts with the OS mechanism, then follows the idea into storage.

CoW in fork()¶

fork() creates a child process that is a copy of the parent: same memory contents, same open files, same code. Physically copying a multi-gigabyte heap would make fork() take seconds. Instead:

The cost of fork() is therefore proportional to the size of the page tables, not the memory — still noticeable for large processes: Redis's latency documentation reports on the order of 10 ms per GB of resident memory on typical Linux hardware with 4 KB pages (and more on some virtual machines), since each PTE must be copied and write-protected while the parent is paused. Each physical frame has a reference count (mapcount); when a write fault finds the count is 1 (the other owner has already copied or exited), it just makes the page writable again without copying.

Measured¶

A parent filled 100 MiB of anonymous memory, then forked (Apple Silicon macOS, 16 KB pages). The child read every page, then wrote to the first 10 MiB:

The writes are the part that matters on every OS: one page copy per page first written, regardless of how many bytes in the page change. A child that modifies one byte in each of 6,400 pages copies all 100 MiB.

fork + exec, vfork, and posix_spawn¶

The common use of fork is to immediately exec a new program, which discards the copied address space. CoW makes that cheap, but copying the page tables of a huge process is still wasted work, and while it runs the parent's memory is write-protected (its next writes all fault). vfork (child borrows the parent's address space until exec) and posix_spawn (which Linux implements with vfork-like semantics) avoid it; launching subprocesses from a large server process should use them.

The Redis snapshot: CoW as a database feature¶

Redis takes point-in-time snapshots (RDB) by calling fork():

1. The child inherits a view of the entire dataset frozen at the moment of the fork — CoW guarantees it never sees later writes.
2. The child serializes that view to disk at its own pace.
3. The parent keeps serving reads and writes. Every page it writes during the snapshot is copied once.

That gives a consistent snapshot with no locks and no pause beyond fork itself. The cost appears as memory:

peak memory ≈ dataset + (number of distinct pages written during the snapshot × page size)

Worked example. A 20 GB Redis instance receives writes spread across its keyspace at a rate that touches 30% of pages during a 60 s snapshot: peak memory ≈ 20 + 6 = 26 GB. If writes touch nearly every page (random updates over a hash table), it approaches 40 GB. Provisioning for steady-state usage alone is how a snapshot triggers an OOM kill.

Two Linux settings matter here, and both appear in Redis's own warnings at startup:

• vm.overcommit_memory = 1: under the default heuristic, fork of a 20 GB process on a 32 GB machine may be refused, since the kernel conservatively worries the child could write everything.
• Transparent huge pages off: with THP, the CoW unit is a 2 MB huge page, so writing one byte copies 2 MB instead of 4 KB — up to 512x more copying, and long latency spikes in the parent.

CoW for private file mappings¶

mmap(..., MAP_PRIVATE, fd, ...) maps a file such that reads come from the page cache, but the first write to a page creates a private anonymous copy; the file is never modified. The loader maps a program's .data segment this way: every process starts with the initial values from the executable, and each gets its own copy only of the pages it modifies.

CoW in storage: never overwrite in place¶

Copy-on-write B+Trees (LMDB, BoltDB)¶

A CoW B+Tree never modifies a page that a reader might see. Updating a leaf writes a new leaf, which requires a new parent pointing to it, and so on up to a new root:

A transaction commits by writing its new pages, syncing, and then atomically updating a small meta page to point at the new root. Consequences:

• No WAL needed: until the meta page switches, the old tree is intact; a crash leaves the old root valid. Recovery is instant.
• Readers never block and never see partial updates: a reader keeps using the root it started with; it is a snapshot for free.
• Write amplification: one key update rewrites height pages (3–4 pages for one small change), mitigated by batching many updates per transaction so the upper pages are shared.
• Garbage: old pages can be reused only when no reader still uses a root that references them. LMDB tracks the oldest active reader's transaction ID for exactly this; a long-lived read transaction stops page reuse and the file grows — the CoW version of MVCC's "oldest snapshot" problem (see MVCC in Storage Engines).

Copy-on-write file systems and snapshots¶

Btrfs, ZFS, and APFS apply the same idea to file system blocks. Since blocks are never overwritten, a snapshot is just keeping an old root alive: it takes constant time and no space initially, and consumes space only as the live file system diverges from it. The same property breaks something databases rely on: a database that overwrites pages in place on a CoW file system gets each page write turned into an allocation of a new block, so files fragment heavily. Running databases on Btrfs commonly involves disabling CoW for their directories (chattr +C).

LSM engines: immutability instead of copying¶

LSM engines never modify an SSTable; a new Version object describes which files exist, and readers hold a reference to the version they started with (see Compaction and Recovery). That is copy-on-write at the level of metadata: change = build a new version, swap one pointer, free the old one when its last reader leaves.

Common pitfall¶

Assuming a fork()-based snapshot is cheap because fork() returns quickly. The bill arrives afterwards, in the parent: every page it writes during the snapshot is copied, so a write-heavy process can approach double its memory, and each first write to a page costs a fault and a page copy, adding latency to whatever request triggered it. With transparent huge pages enabled, each of those copies is 2 MB. Size memory for the peak during snapshots, keep THP off for fork-snapshotting processes, and monitor the copy volume (Redis reports it as rdb_last_cow_size).