CPU and Cache Efficiency¶

The machine you are actually programming¶

Once allocations are under control, performance is decided by how well the code fits the hardware: whether data arrives from L1 cache in a nanosecond or from DRAM in a hundred, whether branches are predictable, whether cores fight over cache lines. Go hides none of this — a Go slice is a contiguous array, a pointer chase is a pointer chase. The hardware background is in Memory Hierarchy and CPU Cache; this page measures what it means for Go code.

Measurements: Go 1.26.4, Apple M1 Pro (128-byte cache lines on the performance cores' L2; 64-byte lines are the norm on x86). Each function under test is //go:noinline and called from b.Loop, so the benchmark harness does not distort it (Benchmarking Methodology).

Memory access patterns dominate¶

Summing 1 M int64 values (8 MB) stored three ways:

Layout	ms per pass	Relative
`[]int64` (contiguous)	0.43	1x
Linked list, nodes allocated in order	1.37	3.2x
Linked list, same nodes, shuffled order	28.4	66x

Same data, same number of additions. The slice streams through memory sequentially; the hardware prefetcher detects the pattern and loads cache lines before they are needed, so the loop runs at memory bandwidth. The in-order list is slower because each step must load a pointer before it knows the next address (a dependent load chain), but consecutive allocations sit close together, so the prefetcher still helps. The shuffled list defeats everything: each node is a cache miss and often a TLB miss, ~27 ns each — main-memory latency.

Go-specific consequences:

[]T beats []*T for iteration, and []*T with objects allocated at different times (scattered across the heap) is the shuffled case.
Maps, linked structures, and trees of pointers have this cost on every lookup. A sorted slice with binary search often beats a map for small, read-mostly sets, and a B-tree beats a binary tree because each node is a contiguous block (Buffer Pool and B+Tree).
The GC's mark phase is also a pointer chase over your heap, which is why Green Tea's span-at-a-time scanning helps (Garbage Collector).

Data layout: arrays of structs vs. structs of arrays¶

Summing one float64 field over 1 M particles:

type particle struct {            // 96 bytes per element
    x, y, z, vx, vy, vz float64
    alive bool
    name  [40]byte
}

Layout	ms per pass	Bytes touched
`[]particle`, read `.x`	1.74	96 MB (every cache line of every struct)
`xs []float64`	0.98	8 MB

The struct-of-arrays version moves 12x less data. It is "only" 1.8x faster here because once the data is small enough, the loop is limited by the latency of floating-point addition (each s += v must wait for the previous one, ~3 cycles); using several independent accumulators would speed it up further. The lesson generalizes: first cut bytes moved, then look at dependency chains. Hot-path structs should keep hot fields together and push cold fields (names, debug info, rarely-used config) into a separate struct behind a pointer.

Branch prediction, and when the compiler removes the branch¶

Summing elements ≥ 128 in 1 M random bytes, sorted vs. unsorted:

Loop body	Random order	Sorted
`if v >= 128 { s += v }`	0.66 ms	0.65 ms	No difference
Same test, plus a counter update with a second branch inside	3.60 ms	1.14 ms	3.2x slower on random data

The classic "sorted data is faster" result appears only in the second loop. In the first, the compiler emitted a conditional select instead of a branch — the disassembly shows CMPW $128, R4 followed by CSEL GE, R5, R3, R3 (Compiler, Inlining, and Assembly) — so there is nothing to mispredict. The second body was too complex to convert, and on random data the branch predictor guesses wrong about half the time, costing ~10–15 cycles per miss.

Practical rules: keep hot-loop bodies simple enough for branchless code generation; separate rare cases (errors, slow paths) into functions so the hot path is short; when data order is under your control (e.g. batch processing), grouping similar items makes branches predictable.

Eight goroutines each incrementing their own atomic counter, 100,000 times each:

type packed struct{ n atomic.Int64 }                  // 8 bytes: 8 counters share a line
type padded struct{ n atomic.Int64; _ [120]byte }     // 128 bytes: one line each

Layout	ms per round
`[8]packed`	53.4
`[8]padded`	1.18	45x faster

The goroutines share no data logically, but the eight counters sit in the same cache line. Each increment needs the line in exclusive state, so the line ping-pongs between cores, every write invalidating the other seven copies. Padding each counter to its own line removes the coherence traffic entirely.

Where it bites in Go:

Per-worker stats structs in a slice ([]workerStats), written by different goroutines.
Sharded locks or counters without padding (the sharded counter in Channels, Mutexes, and the Memory Model is padded for this reason).
Adjacent fields of a struct written by different goroutines — e.g. a producer index and a consumer index in a ring buffer. Put padding between them (_ [128]byte covers both 64- and 128-byte lines; golang.org/x/sys/cpu.CacheLinePad is the portable spelling).

Small collections: linear beats clever¶

Looking up 4 keys among 8 HTTP method names:

Structure	ns per 4 lookups
`map[string]int`	35.1
Linear scan of a `[]string`	19.4

A map lookup must hash the string and probe; a linear scan over 8 short strings compares lengths first and touches one or two cache lines. Below a few dozen elements, a slice scan is usually faster than a map — and sorted-slice binary search usually wins up to hundreds. Always measure with your real sizes; this crossover is exactly the kind of fact that changes with hardware.

Measuring hardware effects directly¶

Timing tells you that something is slow; hardware counters tell you why. On Linux:

perf stat -e cycles,instructions,cache-misses,branch-misses,L1-dcache-load-misses \
    ./pkg.test -test.run='^$' -test.bench=Traverse

IPC (instructions per cycle): above ~2 means the CPU is busy computing; below ~0.5 means it is mostly waiting on memory.
cache-misses per operation map directly to the pointer-chasing and layout costs above.
branch-misses confirm or rule out misprediction.

perf record -g + perf report also profiles Go programs with correct symbols (Go emits frame pointers by default), including time inside the runtime and cgo, which pprof sees less well. More in Linux Observability Tools.

A checklist for a hot loop¶

Is the data contiguous, and is it iterated in memory order?
Are the bytes touched per item minimal (hot fields together, no pointer per element)?
Is there a dependency chain (each iteration waiting on the previous) that independent accumulators could break?
Are branches predictable, or simple enough to become conditional moves?
Are bounds checks eliminated where it matters?
Do goroutines write to memory within the same cache line?
Did you confirm each hypothesis with a counter or a benchmark, rather than by reasoning alone?

Common pitfall¶

Assuming N goroutines give an N-times speedup, without measuring how the work scales. Splitting two loops across goroutines on this 8-core laptop:

Goroutines	Sum a 1 GB `[]int64` (memory-bound)	SHA-256 of 64 MB (compute-bound)
1	60.4 ms	28.5 ms
2	28.3 ms	14.4 ms
4	14.6 ms	7.5 ms
8	11.2 ms (5.4x)	5.4 ms (5.3x)

Neither reached 8x. Both scale almost linearly up to 4 goroutines and then flatten, because the M1 Pro has 6 performance and 2 efficiency cores: the last two goroutines run on slower cores, and the whole operation waits for the slowest chunk. The memory-bound scan scaled as well as hashing only because this chip's memory bandwidth (~200 GB/s) is unusually high relative to its core count; one core alone pulled about 18 GB/s. On a server where many more cores share each memory controller, a bandwidth-bound loop stops scaling much earlier, while compute-bound work keeps scaling with cores.

The habit: run the benchmark with -cpu=1,2,4,8 (or vary the number of workers) and look at the curve before choosing a parallelism level, split work into more chunks than workers so fast cores are not idle while slow ones finish, and check IPC and cache-miss counters to know which kind of loop you have.

Review questions¶

Why is the shuffled linked list 66x slower than the slice when the same additions are performed?
Why was the struct-of-arrays layout only 1.8x faster despite reading 12x less data?
Why did sorting the input make no difference for the first branch loop?
Explain false sharing using the packed-counter result. How would you find it in a production service?
Why can a linear scan beat a map, and how would you find the crossover point for your data?
Why did 8 goroutines give only about 5.4x on an 8-core laptop, and how would you design the work split to do better?