Topic Map

A map of the handbook for looking things up. Every page stands on its own; links inside each page point to the background it relies on, so any page is a valid starting point.

How the areas connect

graph LR GI["Go Internals"] --> PERF["Performance and Benchmarking"] OS["Operating Systems"] --> GI OS --> ST["Storage and Files"] OS --> NET["Networking"] ST --> DB["Databases"] DB --> SE["Build a Storage Engine in Go"] PERF --> SE PERF --> DBG["Debugging and Production"] NET --> DBG SE --> DS["Distributed Systems"] NET --> DS DS --> SD["System Design"]

Areas

Area What it covers
Go Internals Value layout, escape analysis, the GMP scheduler, channels and the memory model, goroutine leaks, the GC, the compiler, unsafe/cgo/syscalls
Operating Systems Processes and threads, system calls, virtual memory and page faults, scheduling, file descriptors, I/O models, synchronization
Performance and Benchmarking Benchmark methodology and statistics, pprof, the execution tracer, allocations, CPU caches, load testing and tail latency
Storage and Files Storage hardware, the Linux I/O stack, fsync and durability, encoding and checksums, caches, compression
Networking TCP/UDP/DNS/TLS, the Go netpoller, TCP servers, binary protocols, HTTP and gRPC, pooling, load balancing, resilience
Databases Schema and SQL, indexes, B-trees and LSM trees, WAL, MVCC, isolation, locking, query optimization
Build a Storage Engine in Go A working LSM key-value store and B+tree, with tests and benchmarks (source in labs/)
Debugging and Production Panics and dumps, Delve, concurrency bugs, Linux observability tools, incident handling
Distributed Systems Partial failure and clocks (HLC), Raft implemented in Go, a replicated KV store with linearizable reads, leases and fencing, testing with simulation and linearizability checking; replication, partitioning, CAP, distributed transactions
System Design The design framework and case studies

Look up by question

Question Pages
Does this allocate? Why is this on the heap? Escape Analysis and Allocation, Memory Layout of Go Values
What happens when a goroutine blocks on I/O, a syscall, or cgo? Runtime Scheduler, Network Programming in Go
Is this concurrent code correct? Channels, Mutexes, and the Memory Model, Concurrency Bugs
Why is RSS much larger than the heap? How do GOGC and GOMEMLIMIT work? Garbage Collector
Was the function inlined? Is there a bounds check? Compiler, Inlining, and Assembly
Is the new version really faster? Benchmarking Methodology
Where does the CPU time go? Profiling with pprof
Slow under load, but the CPU profile looks normal Execution Tracer, Load Testing and Tail Latency
GC takes too much CPU Reducing Allocations and GC Pressure
Why is this loop slow on the same amount of work? CPU and Cache Efficiency, Memory Hierarchy and CPU Cache
When is written data actually safe on disk? Files and Durability in Go, Durability and fsync
How do I frame messages over TCP? Designing a Binary Protocol
Durable log design, group commit Write-Ahead Log
B-tree or LSM? Buffer Pool and B+Tree, Compaction and Recovery
Snapshots, MVCC, lost updates, write skew Iterators, Snapshots, and Transactions, Isolation Levels and Anomalies
How do I test something that must survive crashes or bad input? Testing and Benchmarking
Reading a panic, a goroutine dump, a core dump Panics, Stack Dumps, and Delve
What is the kernel doing to my process? Linux Observability Tools for Go
How do I order events across machines whose clocks disagree? Failure and Time
How does Raft elect a leader, and why do timeouts need randomness? Raft: Leader Election
When is a Raft entry committed? How are divergent logs repaired? Raft: Log Replication
Snapshots, membership changes, pre-vote, check-quorum, WAL storage for Raft Raft in Production
What does a durable replicated write cost on real disks? Why batching matters Raft on Real Hardware
Exactly-once writes, linearizable reads, ReadIndex Replicated Key-Value Store
How do range-partitioned systems split, move, and route data across many Raft groups? Multi-Raft and Sharding
How do transactions stay atomic across shards? Snapshot isolation, 2PC without a blocking coordinator Distributed Transactions with Percolator
Is a distributed lock safe? Leases and Fencing
How do I test a distributed system and prove linearizability? Testing Distributed Systems
Memory growth, hangs, fd exhaustion, latency spikes in production Production Incident Playbook, Goroutine Leaks

Runnable code

labs/ contains the code behind the Go pages, compiled and tested with the race detector:

Package Pages
labs/kv — LSM key-value store Build a Storage Engine in Go
labs/bptree — B+tree with a buffer pool Buffer Pool and B+Tree
labs/proto — framed, multiplexed protocol Designing a Binary Protocol
labs/raft — Raft with a simulated network Raft: Leader Election, Raft: Log Replication
labs/raftwal — write-ahead log storage for Raft Raft in Production
labs/raftnet, labs/cmd/raftkvd, labs/cmd/kvbench — Raft over TCP, a server process, a load generator Raft on Real Hardware
labs/raftkv — replicated KV store on Raft and labs/kv Replicated Key-Value Store
labs/multiraft — range-partitioned Multi-Raft with splits, replica moves, and replica GC Multi-Raft and Sharding
labs/lincheck — linearizability checker Testing Distributed Systems
labs/hlc, labs/lease — hybrid logical clocks, leases and fencing Failure and Time, Leases and Fencing

Measurements on the Go pages come from Go 1.26 on the hardware named on each page. Re-running them on other hardware gives different numbers; the mechanisms behind them are what carry over.

Further reading

  • The Go Memory Model and the Go runtime source (runtime/proc.go, runtime/mgc.go, runtime/chan.go) — readable, heavily commented.
  • Systems Performance (2nd ed.), Brendan Gregg — methodology and Linux tools.
  • Designing Data-Intensive Applications, Martin Kleppmann.
  • Database Internals, Alex Petrov.
  • Source code: Pebble (LSM engine in Go), etcd/raft, bbolt (B+tree in Go), VictoriaMetrics.