Cards/CachingSystem Design · Day 97Nov 7, 2026

Every Cache Is A Bet On Staleness

Four caches from earlier days — the shopping cart, a catalogue cached at the ISP, stale-while-revalidate and semantic caching — each written out as the bet it makes, with a reality check on freshness traded for latency, load, cost or experience

A cache is never really about memory. It is a wager: this will still be true when someone reads it.

Four caches from the last hundred days, four different bets. The cart held in memory bets that a resurrected item costs less than a failed write. Tonight's catalogue pushed into a box inside the ISP bets we know what people will want tomorrow. Stale-while-revalidate returns the expired value instantly and recomputes quietly behind it, betting that slightly old beats ten thousand identical queries at once. Semantic caching answers a question because it resembles one already answered, betting that "close enough" means the same thing.

Every one of them trades freshness for something better — latency, load, cost, experience. The trade is the point, and it is worth saying out loud, because a cache added without naming the bet is a correctness decision made by accident.

Similarity is the riskiest of the four. The other three can be wrong about when. That one can be wrong about what, because close and the same are different claims.

We're not choosing whether to serve something slightly wrong. We're choosing how wrong, to whom, and for how long.

We're not choosing whether to serve something slightly wrong. We're choosing how wrong, to whom, and for how long.