Caching

What is it: A cache is a fast, often in-memory, store that holds frequently-accessed data to avoid repeatedly computing or fetching it from a slower source (database, external API).

Why: Database queries involve disk I/O, lock acquisition, and result set deserialization — even a fast query takes 1–10ms. A Redis GET takes ~0.1ms. For read-heavy workloads with hot data (e.g., product catalog, user profile), caching reduces database load by 90%+ and latency by 10–100×.

graph TD req["Request for key X"] --> check{"In cache?"} check -->|"hit"| ret["Return cached value<br/>(~0.1ms)"] check -->|"miss"| db["Query database<br/>(~1-10ms)"] db --> store["Store in cache"] store --> ret2["Return value"]

Cache-aside (lazy loading): The application checks the cache first. On a cache hit, return the cached value. On a miss, query the database, store the result in the cache, return the result. The cache is populated lazily — only accessed data is cached.

Trade-offs: first request for each key is slow (cache miss + DB query). If many keys expire simultaneously, a thundering herd of requests all miss the cache and hit the DB at once. Mitigate with staggered TTLs (add random jitter: TTL = base + rand(0, 20%×base)).

Write-through: On every write, update both the cache and the database synchronously. The cache always reflects the current state for recently-written keys.

Trade-offs: write latency increases (two writes instead of one). Cache is polluted with data that is written but never read. Mitigate by combining with TTL — cached entries expire if not read.

Write-behind (write-back): Write to cache only; asynchronously flush to database in batches. Very fast writes. High risk: if the cache fails before flushing, writes are lost. Appropriate only for data that can tolerate some loss (counters, view counts).

Cache invalidation: The hardest problem in caching. When underlying data changes, the cache must be updated or invalidated. Strategies:

  • TTL expiration: simplest; accept eventual staleness up to TTL duration.
  • Event-driven invalidation: on DB write, publish an event; cache service subscribes and deletes the key. Consistent but complex.
  • Write-invalidate: delete the cache key on write rather than updating it. Simpler than write-through; the next read repopulates. Slightly higher miss rate but avoids stale-write race conditions.

When cache becomes stale: A write updates the DB but not the cache (write-aside pattern). Until the cached key expires, reads return the old value. This is acceptable for low-consistency requirements (user profile, catalog) but not for financial balances or inventory counts.

Thundering herd (cache stampede): A popular key expires. Hundreds of concurrent requests all get a cache miss and all query the DB simultaneously. The DB is overwhelmed. Solutions: mutex (first request holds a lock, others wait for cache repopulation); probabilistic early expiration (each request has a small probability of refreshing before expiry, based on remaining TTL and estimated compute time); background refresh (a separate goroutine refreshes the cache before expiry, serving stale-but-not-expired data during refresh).

Real-world usage: Proxel uses Redis as a cache for proxy scores. Scores are computed from the last N requests — expensive to recompute on every job dispatch. Cache the score with a 60-second TTL; accept that scores are slightly stale. On proxy update, invalidate the cache key immediately.

Common pitfall

Invalidating a cache on UPDATE but forgetting the same invalidation on DELETE leaves a deleted row's cached value looking perfectly valid forever (or until its TTL, if one exists) — the application correctly stops writing new data for that key, but a stale read of data that shouldn't exist at all is a more confusing bug to trace than a stale read of data that just changed, because nothing about the symptom points at "this was deleted." Every code path that removes a row needs the same cache-invalidation discipline as the code path that updates one, not just the more obviously-remembered case.