Memory Layout Deep Dive¶
What a process's memory actually contains¶
When a program runs, the operating system gives it a virtual address space (see Virtual Memory) and fills parts of it with distinct segments, each with its own purpose, permissions, and lifetime rules. Knowing which segment a piece of data lives in tells you how long it lives, who frees it, what it costs to allocate, and what kind of bug touching it wrongly produces.
| Segment | Contains | Permissions | Lifetime | Allocated by |
|---|---|---|---|---|
| Text | Machine code | read + execute | Whole process | Loader (mapped from the executable file) |
Read-only data (.rodata) |
String literals, const tables, vtables |
read | Whole process | Loader |
Data (.data) |
Globals/statics with a non-zero initializer | read + write | Whole process | Loader (copied from the file on first write) |
BSS (.bss) |
Globals/statics initialized to zero | read + write | Whole process | Loader (zero pages, nothing stored in the file) |
| Heap | Dynamically allocated memory (new, make, escaping values; malloc in C) |
read + write | Until freed / collected | Allocator, using brk or mmap |
| Memory mappings | Shared libraries, mmaped files, the Go runtime's heap arenas, thread stacks |
varies | Until munmap |
mmap |
| Stack (main thread) | Function frames: locals, return addresses, saved registers | read + write | Until the function returns | The CPU's stack pointer |
| Kernel space | The kernel's own memory | none from user mode | — | — |
Seeing it: a measured layout¶
This Go program prints one address from each region (Go 1.26.4, Apple Silicon, macOS, 64-bit):
var globalInit = 42 // initialized global
var globalZero int // zero global
var sink *[64]byte
var bigSink []byte
//go:noinline
func stackAddr() uintptr {
var local int
return uintptr(unsafe.Pointer(&local)) // only the address escapes, not local
}
func main() {
sink = new([64]byte) // small heap object
bigSink = make([]byte, 64<<20) // large heap object
s := "hello"
fmt.Printf("text (func main) %#x\n", funcAddr(main)) // entry PC of main
fmt.Printf("rodata (string literal) %#x\n", uintptr(unsafe.Pointer(unsafe.StringData(s))))
fmt.Printf("data (initialized var) %#x\n", uintptr(unsafe.Pointer(&globalInit)))
fmt.Printf("bss (zero var) %#x\n", uintptr(unsafe.Pointer(&globalZero)))
fmt.Printf("heap (new([64]byte)) %#x\n", uintptr(unsafe.Pointer(sink)))
fmt.Printf("heap (make 64 MiB) %#x\n", uintptr(unsafe.Pointer(&bigSink[0])))
fmt.Printf("stack (goroutine local) %#x\n", stackAddr())
// ... and stackAddr() called from a second goroutine
}
page size: 16384
text (func main) 0x102195e20
rodata (string literal) 0x102196b25
data (initialized var) 0x1022744a8
bss (zero var) 0x1022a5030
heap (new([64]byte)) 0x46ddbfb9c040
heap (make 64 MiB) 0x46ddbfc00000
stack (goroutine local) 0x46ddbfb90b90
stack (another goroutine)0x46ddbfbc2790
What the output shows:
- Text, read-only data, data, and BSS sit together in the executable's
mapping, in that order — they all come from the executable file.
String literals live in read-only data, which is why converting a
string to a
[]bytewithunsafeand writing to it crashes (unsafe, cgo, and System Calls). - Both heap objects, small and large, live in the Go runtime's own
heap arenas — large
mmaped regions at addresses far away from the executable — not in a traditionalbrkheap. The runtime reserves address space in 64 MB arenas and carves spans out of them (Memory Allocator). - Goroutine stacks are in the same heap region. The locals of
mainand of a second goroutine sit a few hundred KB away from the heap objects: the Go runtime allocates goroutine stacks from its own memory, not from OS thread stacks. Each OS thread (M) still has its own system stack for the runtime and cgo, but Go code runs on goroutine stacks. - The page size here is 16 KB, not the 4 KB assumed in most
textbooks: Apple Silicon uses 16 KB pages; x86-64 Linux uses 4 KB
(with optional 2 MB / 1 GB huge pages); some ARM64 Linux kernels use
16 KB or 64 KB. Code that hard-codes 4096 for alignment or
O_DIRECTbuffers is wrong on such systems — ask withos.Getpagesize(). - The classic textbook picture — heap growing up from low addresses, stack growing down from just below the kernel — describes a C process on Linux x86-64. The segments are universal; their placement is an OS and runtime decision. With ASLR (address space layout randomization) every run gets different addresses, so an attacker cannot predict where code or data lives.
On Linux, cat /proc/<pid>/maps (or pmap -x <pid>) lists every
mapping with its address range, permissions, and backing file; on
macOS, vmmap <pid>. A typical Linux x86-64 map of a static Go binary, abbreviated:
00400000-00b9c000 r-xp /usr/local/bin/kvserver ← text (static Go binary)
00b9c000-0127e000 r--p /usr/local/bin/kvserver ← rodata
0127e000-012c2000 rw-p /usr/local/bin/kvserver ← data + bss
c000000000-c004000000 rw-p ← Go heap arena (goroutine stacks live here too)
7f3e1c000000-7f3e1c400000 rw-p ← runtime metadata, thread stacks (mmap)
7ffd5e8f1000-7ffd5e912000 rw-p [stack] ← main thread's system stack
r-xp = readable, executable, private. A program trying to write to
text, or execute data, gets a segmentation fault: the permissions are
enforced by the page tables (the NX/execute-never bit), which is a
core defense against code-injection attacks.
The stack in detail¶
What a frame holds¶
Each function call pushes a stack frame:
higher addresses
┌──────────────────────────┐
│ caller's frame │
├──────────────────────────┤
│ arguments beyond regs │ (most args are passed in registers)
│ return address │ ← pushed by `call` (x86) / kept in LR (ARM)
│ saved frame pointer │ ← frame pointer chain: used by debuggers/profilers
│ saved callee-saved regs │
│ local variables │
│ spill slots, alignment │
├──────────────────────────┤ ← stack pointer (SP)
│ (free) │
lower addresses
Allocating a frame is one instruction (sub sp, sp, #N) and freeing it
is one more — the reason stack allocation is essentially free, and why
stack memory is always "freed" at exactly the right time.
Limits and guard pages¶
The main thread's stack is limited (ulimit -s; 8 MB by default on
Linux, 8 MB on macOS). Other threads get stacks allocated with mmap
when created (glibc default: equal to the ulimit -s value; macOS: 512
KB for secondary threads). Below each stack is a guard page with
no permissions; running into it raises SIGSEGV instead of silently
overwriting whatever memory lies below.
Worked example — stack growth. A recursive function keeps a 1 KB
local array per frame and recurses 10,000 deep — roughly 10 MB of
stack. A C thread with a fixed 8 MB stack hits the guard page and
crashes with SIGSEGV. In Go it succeeds; measured, the goroutine
stacks in use grew from 352 KiB to 16,736 KiB during the call. A
goroutine starts with a small stack (2 KB minimum; Go 1.19+ starts new
goroutines at the average size observed so far); when it fills, the
function prologue's stack check fails, the runtime allocates a stack
twice as large, copies the frames, and fixes up every pointer into the
stack — possible because the compiler records where every pointer is.
The limit is 1 GB per goroutine on 64-bit systems, after which the
program dies with goroutine stack exceeds 1000000000-byte limit.
Practical consequences in Go:
- Deep recursion is safe up to a point, but each doubling copies the
whole stack: a hot path that recurses deeply in a fresh goroutine pays
runtime.morestack/copystackrepeatedly (visible in CPU profiles). - Deep recursion over user-controlled input (nested JSON, a tree of unknown depth) can still be turned into memory exhaustion by an attacker; bound the depth or use an explicit stack (a slice) in a loop.
- Returning a pointer to a local is safe: the compiler moves such variables to the heap (Escape Analysis and Allocation). The cost is an allocation, which is why it matters in hot paths.
- cgo calls run on the thread's system stack, which has a fixed size; deep recursion in C code called from Go does crash.
Frame pointers and stack traces¶
Profilers and debuggers walk the chain of saved frame pointers to
reconstruct call stacks. Many C compilers omit the frame pointer when
optimizing (one more free register), so their stack traces need DWARF
unwind information, which is slower and less reliable in sampling
profilers. The Go compiler keeps frame pointers on amd64 and arm64, so
perf and other sampling profilers get correct Go stacks without
extra flags (see
Linux Observability Tools for Go).
The heap in detail¶
Where heap memory comes from¶
The allocator (see Memory Allocator) obtains memory from the kernel in two ways:
brk/sbrkmoves the end of the traditional heap segment up or down. It can only shrink from the top, so a single live object at the top pins everything below it.mmap(MAP_ANONYMOUS)creates an independent region anywhere in the address space;munmapreturns it to the OS whole. Large allocations and modern allocators' arenas use this.
Demand paging: the heap is lazy¶
Neither call allocates physical memory. They reserve virtual address
range; physical pages are assigned one at a time on the first touch
(a page fault). Measured from Go on the same machine,
with unix.Mmap and the fault counter from getrusage:
| Step | Minor page faults |
|---|---|
unix.Mmap 1 GiB anonymous |
+0 |
| Write one byte per page across the first 100 MiB | +6,400 (= 100 MiB / 16 KiB) |
Exactly one fault per page, and resident memory grows only by what was touched. Consequences:
- A process's virtual size (VSZ) can be enormous while its resident set (RSS) is small; only RSS uses RAM.
- The first pass over freshly allocated memory is slow — every page
costs a fault (~1–2 µs including zeroing). This is why a Go program
with
GOGC=offthat allocates gigabytes was slower than one that collects and reuses memory (Garbage Collector). Latency-sensitive engines pre-touch buffers at startup, or map withMAP_POPULATEon Linux. - A successful allocation does not guarantee memory: under Linux overcommit, the failure arrives later as the OOM killer when pages are actually touched.
Fragmentation¶
Worked example. A process allocates 1,000 objects of 100 bytes (100 KB total), then frees every other one — 500 scattered 100-byte holes now sit in the free list, non-contiguous. A later 300-byte allocation fits in none of them, even though 50 KB is technically free, so the allocator requests fresh memory. Go's size-class allocator limits this (objects of similar size share spans), but a span is only returned when every object in it is dead, and the GC never moves objects to compact them. Long-running processes with many alloc/free cycles of varying sizes therefore show RSS creeping upward even when the live object count is stable — the freed memory exists, but not in a usable shape, and pages containing even one live object cannot be returned to the OS.
Where Go values end up¶
| Declaration | Where |
|---|---|
x := 0 inside a function, address never escapes |
Stack (or only a register) |
p := &T{} returned or stored globally |
Heap ("escapes to heap") |
var counter int at package level |
BSS (.noptrbss if it contains no pointers) |
var table = []int{1, 2, 3} at package level |
Slice header in data; elements in data |
s := "literal" |
String header on the stack; bytes in read-only data |
b := make([]byte, 64) that doesn't escape |
Stack (constant size up to 64 KB) |
b := make([]byte, n) with non-constant n |
Heap, unless n ≤ 32 (Go 1.25+ uses a stack buffer) |
| Captured variable of a closure that outlives the function | Heap |
| Goroutine stack | Go heap arena, managed by the runtime |
The header and the memory it points to are often in different places:
a slice header is 24 bytes wherever it lives; the elements are
elsewhere. go build -gcflags=-m prints the compiler's decision for
each allocation (Escape Analysis and Allocation).
Common pitfall¶
Reading RSS (or top's memory column) as "how much memory my data
uses." RSS includes shared library pages also counted by every other
process using them, file-backed pages that the kernel can drop at any
moment, and heap pages that hold only freed-but-not-returned memory;
it excludes memory that was reserved but never touched, and swapped-out
pages. For a database process the honest breakdown is the allocator's
own statistics (for Go, runtime/metrics: live heap vs. heap retained
vs. released to the OS), plus the engine's cache accounting, plus the kernel's view per mapping (/proc/<pid>/smaps:
Rss, Pss, Private_Dirty, Swap for each region). A "memory leak"
diagnosed from RSS alone is, as often as not, fragmentation or page
cache.