Concurrency Primitives¶

What they are for¶

When threads share data, two things must be controlled: mutual exclusion (only one thread modifies the data at a time, so it never sees a half-updated state — see Race Conditions and Critical Sections) and coordination (a thread waits until another has produced something). Every primitive on this page is a combination of those two, built from the same hardware foundation: atomic read-modify-write instructions (see Memory Model and Atomics) plus the kernel's ability to put a thread to sleep and wake it.

Understanding how they are built explains what they cost, which is what decides which one to use.

Measured costs¶

All numbers below: Go 1.26.4, Apple M1 Pro (8 cores). "ns/op" is total wall time divided by total operations across all goroutines — the inverse of aggregate throughput. Each operation increments or reads a shared counter, i.e. a critical section of a few nanoseconds: the worst case for synchronization overhead.

Four lessons are already visible:

1. Uncontended locking is cheap (~13–16 ns): a mutex nobody else wants is one atomic compare-and-swap and never involves the scheduler.
2. Contention is what costs. With 8 goroutines on one counter, the mutex costs 9x its uncontended price and even the atomic costs 10x: every operation moves the cache line to another core.
3. Not sharing beats every primitive: per-goroutine counters scaled linearly (0.40 ns/op aggregate at 8 goroutines, ~170x faster than the atomic). Sum them on read.
4. A reader-writer lock is not automatically faster for readers: for short critical sections, RWMutex.RLock cost the same as a plain Mutex, because acquiring a read lock still writes a shared reader count.

The spinlock row looks like a clear winner and is explained in the next section — it is mostly a lesson about what this benchmark does not measure.

Spinlocks¶

The simplest lock is a flag set with an atomic swap:

type SpinLock struct{ locked atomic.Bool }

func (l *SpinLock) Lock() {
    for {
        if !l.locked.Swap(true) {
            return // got it
        }
        for l.locked.Load() { // wait by reading, not writing
            runtime.Gosched()
        }
    }
}

func (l *SpinLock) Unlock() { l.locked.Store(false) }

The inner loop waits on a plain load ("test-and-test-and-set"): all waiters read their cached copy of the line, and only when the holder releases do they race to Swap. Spinning on Swap directly would write the line on every iteration and flood the interconnect.

In the table this spinlock was 8x faster than sync.Mutex under contention, and just as fair (8 goroutines grabbing it for 1 s: 6.7–7.3 million acquisitions each, against 1.24–1.29 million for the mutex). It wins because waiters never park: Gosched puts the goroutine back on the run queue, whereas a contended sync.Mutex eventually parks waiters and must wake them, a scheduler round trip on each handoff.

What the micro-benchmark hides is what waiting costs everyone else. Second experiment: 8 goroutines take a lock and hold it for 20 µs of work, while 8 other goroutines do independent CPU work, for 2 s:

A waiter that spins without yielding holds its P (one of the 8 slots that run Go code) for up to 10 ms until the runtime preempts it (Runtime Scheduler). Seven spinning waiters took most of the machine away from both the lock holder and the unrelated work. With Gosched, waiting cost little here because there was always other work to yield to — but every waiter still burns CPU cycles polling, which shows up the moment the machine is otherwise busy or the critical section gets long.

Spinlocks are right only when the critical section is shorter than a context switch and the holder is guaranteed to be running. The kernel uses them because it can disable preemption while holding one. In Go, use sync.Mutex, which already spins briefly (when other Ps are running) before it parks — the adaptive middle ground.

Mutexes and futexes¶

A mutex must sleep under contention, but sleeping requires the kernel, and a system call on every lock would cost ~100+ ns (System Calls). Linux's futex ("fast user-space mutex") solves this: the lock is an integer in user memory, and the kernel is involved only when there is a waiter.

state: 0 = unlocked, 1 = locked, 2 = locked and someone may be waiting

lock():
  if CAS(state, 0 → 1) succeeds: return              ← fast path, no syscall (~10 ns)
  spin a little, retrying the CAS                     ← adaptive spinning
  loop:
    if exchange(state, 2) == 0: return                ← got it (we marked "contended")
    futex_wait(&state, expected=2)                    ← sleep in the kernel while state == 2

unlock():
  if exchange(state, 0) == 2:                         ← there may be waiters
    futex_wake(&state, 1)                             ← syscall only in this case

futex_wait(addr, expected) atomically checks that *addr still equals expected before sleeping, so a wakeup between the user-space check and the sleep cannot be lost. glibc's pthread_mutex works this way, and macOS has equivalent ulock calls. Go's sync.Mutex uses the same three-state idea but parks the goroutine in the Go scheduler instead of sleeping the thread: the M runs other goroutines while the waiter is parked, and the Go runtime itself uses futexes only when an entire thread has nothing to do.

Fairness and convoys¶

A plain futex mutex is not fair: a thread that just released the lock can re-acquire it before a sleeping waiter wakes up (waking takes microseconds). That is good for throughput, but a waiter can starve. Fair (FIFO) locks hand the lock directly to the next waiter and pay a wakeup on every handoff; under contention they can be 10–100x slower. Go's sync.Mutex does both: normally unfair, it switches to starvation mode (direct FIFO handoff) when a waiter has waited more than 1 ms (Channels, Mutexes, and the Memory Model). When a lock is held while the holder is descheduled, every thread that wants it queues behind it — a lock convoy — another reason to keep critical sections short and never do I/O while holding a hot lock.

Reader-writer locks¶

A reader-writer lock (Go's sync.RWMutex, pthread_rwlock) allows many concurrent readers or one writer.

As measured, its value depends on the length of the read-side critical section: acquiring a shared lock is an atomic write to a shared counter, so with tiny critical sections it costs more than a mutex. It pays off when readers hold the lock long enough that parallel reading matters (walking a large structure, copying a big value).

Policy questions every rwlock implementation answers:

Go's RWMutex is writer-preferring: a pending Lock blocks new RLocks, so a goroutine that calls RLock twice (recursively) can deadlock if a writer arrives between the two calls.

Worked example — picking the right primitive. An in-memory config cache is read by 1,000 requests per second and replaced by a background refresher every 30 seconds. A mutex serializes all readers; an rwlock lets them proceed in parallel but still makes each one write a shared reader count. The best option avoids locking readers entirely: keep the config in an immutable object behind an atomic.Pointer[Config] (the copy-on-write/RCU pattern — see Lock-Free Programming and Memory Reclamation). Readers load the pointer; the refresher builds a new config and swaps the pointer. Readers never block and never contend.

Semaphores¶

A semaphore is a counter with two operations: acquire (wait until the count is positive, then decrement) and release (increment, waking a waiter). A semaphore with count N limits concurrency to N.

var ioSlots = make(chan struct{}, 16) // at most 16 concurrent disk reads

func readBlock(f *os.File, off int64, buf []byte) (int, error) {
    ioSlots <- struct{}{}        // acquire (blocks when 16 are in flight)
    defer func() { <-ioSlots }() // release, on every return path
    return f.ReadAt(buf, off)
}

A buffered channel of capacity N is Go's idiomatic counting semaphore; golang.org/x/sync/semaphore adds weighted acquisition and context cancellation. Typical uses: limiting outstanding I/O (blocking file I/O holds an OS thread per call — Runtime Scheduler), connection pools, admission control. Unlike a mutex, a semaphore has no owner — any goroutine may release — which makes ownership bugs (a missed release on an error path) easy to write. Release with defer right after acquiring.

Condition variables¶

A condition variable lets a goroutine sleep until some state protected by a mutex changes. In Go it is sync.Cond:

var mu sync.Mutex
var nonEmpty = sync.NewCond(&mu)

func take() Item {
    mu.Lock()
    defer mu.Unlock()
    for len(queue) == 0 { // loop: re-check after every wakeup
        nonEmpty.Wait()   // atomically unlocks mu and sleeps; relocks before returning
    }
    it := queue[0]
    queue = queue[1:]
    return it
}

func put(it Item) {
    mu.Lock()
    queue = append(queue, it)
    mu.Unlock()
    nonEmpty.Signal() // wake one waiter; Broadcast wakes all
}

Wait atomically releases the mutex and sleeps, and re-acquires it before returning — so the condition is always checked under the lock. The loop is mandatory: between a Signal and the waiter re-acquiring the mutex, another goroutine may have consumed the state. Broadcast is needed when waiters wait for different conditions on the same Cond. Most Go code uses channels instead; Cond remains the cheapest way to wake all waiters on a state change, which is why the storage engine's write stall uses it (Channels, Mutexes, and the Memory Model).

Barriers, latches, and once¶

Atomics as primitives¶

For a single word of state — a counter, a flag, a pointer — an atomic operation is the primitive itself: Add, Swap, CompareAndSwap. They never sleep and never deadlock, but they coordinate only one variable. Once an invariant spans two variables ("the count matches the list length"), you need a lock or a lock-free algorithm designed for it. See Memory Model and Atomics.

Channels and message passing (Go)¶

A Go channel (hchan) is a mutex-protected ring buffer plus queues of waiting senders and receivers. When a receiver is already waiting, a send copies the value directly to it and makes it runnable; blocked goroutines are parked by the Go scheduler, not by the kernel, so blocking is cheaper than an OS thread sleep (see Runtime Scheduler). Closing a channel wakes all receivers. Channels are a higher-level primitive — transfer of ownership — built on the same mutex-and-wakeup machinery.

Choosing¶

Common pitfall¶

Reaching for a "faster" primitive instead of reducing sharing. Replacing a mutex with an atomic, or with a reader-writer lock, often helps less than expected — in the measurements above the contended atomic was still 10x its uncontended cost, and the read lock cost exactly as much as an exclusive one — because the real cost is a cache line bouncing between cores, which every primitive on a shared variable pays. The changes that actually scale are structural: shard the data (per-goroutine or per-core state, lock striping), shorten critical sections (no I/O, no allocation while holding a lock), or make the shared state immutable and swap it atomically.