Cache invalidation: reads, writes, and the stale window

Problem

Every cache I have shipped had the same embarrassing property: the moment a write landed, I could not say with a straight face what the next read would return. The old value lingered for some TTL-shaped while, served with full confidence and zero indication that the origin had already moved on. Nobody gets paged for a stale read — there is no error, no status code, no log line that says "you just showed the customer yesterday's price." The failure is silent, which is exactly why it survives every code review. I built this demo so the silence becomes visible: one SKU, one cache, and a counter that climbs every time a read serves a value the origin no longer holds.

Context

The setup is deliberately the smallest system where the problem is real. A single SKU value lives in SQLite as the origin of truth, fronted by an in-process cache entry with a configurable TTL; every read and write goes through HTTP so I can watch the exact path each request takes. The demo exposes three controls: a cache on/off toggle, a strategy toggle between invalidate-on-write and TTL-only, and a TTL input from 0 to 3600 seconds. Each read returns its value, its source, the milliseconds it took, and a step-by-step flow readout of request to cache to origin. A stats panel tracks hits, misses, writes, and the number that matters most — stale-served. Nothing here is exotic on purpose: this read-through shape is what most product caches look like before someone adds pub/sub invalidation, and its stale window is the one I keep re-learning about.

Alternatives Considered

The first alternative I weighed was no cache at all: every read goes straight to SQLite. It is trivially correct — no cached copy, no stale window — and for a single-row lookup the origin read is already fast. I rejected it because it answers the wrong question: the demo exists to show the cost of caching, and you cannot feel a trade-off with only one side on the table. The cache-off toggle survives in the UI as the baseline: with the cache off, every read I measured cost around a quarter of a millisecond against the origin, which is the price freshness charges per request.

The second alternative was versioned reads: stamp every value with a monotonic version and let readers detect staleness themselves. It is the honest answer where readers tolerate retries, and the write path never learns the cache topology. I rejected it because it pushes version-comparison logic and retry policies onto every reader, scattering the interesting behavior across clients instead of one place I can point at. Invalidation keeps the reader contract simple — what you get is fresh — and makes the writer pay the coordination cost.

The third option was a short fixed TTL with no strategy toggle — the classic "cache everything for 60 seconds and hope" most teams ship first. It bounds staleness with zero write-path work. I kept its spirit as the TTL-only strategy but refused to make it the only mode, because a fixed TTL turns staleness into weather rather than a decision. The toggle is the whole point: the same write produces a stale read in one mode and a fresh one in the other.

Solution

I built the demo as a single Go binary with SQLite. The core is a small cache struct guarding one entry (value plus fill timestamp), a strategy flag, a TTL, and four counters. Reads check the entry first: if the cache is on and the entry's age is under the TTL, the stored bytes are served immediately, otherwise the origin is read and the entry refilled. Writes commit to SQLite and then apply the strategy — invalidate-on-write drops the entry, TTL-only keeps it. One deliberate piece of instrumentation makes the demo work: after serving a hit, the core consults the origin once more, purely to learn whether what it just served matches. A production cache cannot know it is stale without asking; the oracle read is what promotes the stale window from silent to counted, and the flow readout says so out loud.

The hot path is short enough to show in full:

if c.entry != nil && c.now().Sub(c.entry.filledAt) < c.ttl {
    served := c.entry.value
    c.hits++
    current, err := c.st.Get() // oracle check: instrumentation only
    if err != nil {
        return ReadResult{}, err
    }
    stale := current != served
    if stale {
        c.stale++
    }
    return ReadResult{Value: served, Source: SourceHit, Stale: stale, ...}, nil
}
v, err := c.st.Get() // MISS: pay the origin trip, refill the entry

You can run the whole story yourself from the repo. Boot it with DB_PATH=/tmp/cache-demo.db PORT=8123 go run ., set TTL-only with a 3-second TTL, and read once: a miss refilling v1 in 0.265 ms. Write v2, read again, and the response is a hit serving the old v1 in 0.001 ms, the flow readout ending in "STALE (stale-served #1)" — roughly 370 times faster than the origin trip, and wrong. Wait past the TTL and read once more: the entry is 7.842 seconds old against a 3-second TTL, so it misses and refills fresh v2 in 0.220 ms. Flip to invalidate-on-write, write v3, and the next read is a miss serving v3 — session counters were 1 hit, 3 misses, 2 writes, stale-served stuck at 1. POST /reset restores the seed (v1, TTL-only, 5-second TTL, zeroed counters). The behavior tests drive all of this over HTTP with an injectable clock, asserting expiry by advancing time 61 seconds past a 60-second TTL — the full suite passes with go test -race in about a second, no sleeps, no flakes.

Trade-offs

Invalidate-on-write buys freshness with writer complexity. Every write path must know about every cache that might hold the value; miss one replica or region and the stale window reopens exactly where you stopped looking. In this demo invalidation is one nil assignment — in a fleet it is fan-out with partial failure, retries, and ordering questions. Writes also pay latency they did not pay before, and a hot key under write-heavy traffic can spend its life invalidated, turning the cache into a pass-through with extra steps. I pay that gladly where staleness is a correctness bug (prices, balances, permissions), and reluctantly everywhere else.

TTL-only buys simplicity with a bounded lie, and the bound is only as good as the TTL choice. A 3-second TTL means any read can be up to 3 seconds out of date — fine for a product description, unacceptable for inventory. Shorter TTLs shrink the window but raise the miss rate, loading the origin; a zero TTL, which the demo allows, is just the no-cache baseline wearing a costume. The failure mode I dislike most is correlation: writes cluster during launches and incidents, so the stale window opens widest exactly when freshness matters most.

The demo itself trades realism for legibility in two ways I want to name. The oracle check means every demo hit performs one extra origin read a real cache would skip, so the measured 0.001 ms hit time slightly understates how cheap a true hit is. And an in-process entry has no network hop or serialization, so the 370x gap I measured flatters caching relative to a Redis round-trip. Both simplifications understate the performance case while making the correctness case unmistakable — the bias I want when the lesson is "invalidation is a correctness strategy first."

Production Notes

What changes when this leaves the demo: the cache stops being a struct field and becomes shared infrastructure, and every simplification above sends a bill. A networked cache adds pooling, timeouts, and a failure mode the demo never exercises — origin reads failing while the cache holds a stale-but-servable copy, forcing the serve-stale-or-error decision explicitly. I would add a stale-serving policy with its own metric, because "degraded but answering" beats "correct but down" for most read paths, as long as the staleness is labeled.

Monitoring is where I would spend the first production hour. The four demo counters map directly to the dashboards that matter: hit ratio per key prefix, stale-served rate with an alert threshold expressed as a budget rather than zero, and entry age distribution to catch TTL misconfiguration. I would also log the strategy per write, because the incident version of this demo is always "someone deployed a path that writes without invalidating," and per-path attribution turns that from archaeology into a query.

Two scaling hazards deserve explicit mention. First, refill storms: when a hot entry's TTL lapses, every concurrent read misses at once and hammers the origin — the demo's sequential reads hide this completely, and production needs single-flight refills or jittered TTLs. Second, sizing: the demo seeds one row and caps events at 50 in memory, which is correct for disposable demo data and wrong everywhere else — a bounded eviction policy is the first thing I would add before any real dataset touches this shape.