Caching

A cache stores the result of expensive work so you can reuse it instead of redoing it — the single most effective performance tool there is. The performance-and-tradeoffs lesson named its cost; here we go deeper, because caching done carelessly creates bugs that are maddening to track down.

What caching buys

Caching trades memory and freshness for speed. Keep frequently-needed, expensive-to-produce data close and ready:

• A database query result kept in memory.
• A computed value reused instead of recalculated.
• A CDN serving static files from a location near the user.

When the same expensive thing is needed repeatedly and changes rarely, a cache is a massive win.

The hard part: invalidation

"There are only two hard things in computer science: cache invalidation and naming things."

The joke is real. The moment the underlying data changes, the cache is stale — serving the old value. Deciding when to refresh is genuinely hard. Common strategies:

• TTL (time to live): entries expire after a set time. Simple; tolerates brief staleness.
• Write-through / explicit invalidation: update or clear the cache whenever the source data changes. Fresher, but you must catch every write path — miss one and you serve stale data forever.

There's no free lunch: you're always choosing how much staleness you can tolerate against how much complexity you'll accept to reduce it.

Common pitfalls

• Caching too aggressively — users see outdated data and you get "it's wrong but only sometimes" bug reports (often really a stale cache).
• The thundering herd / stampede — many requests miss a just-expired entry at once and all recompute it together, hammering the source.
• Caching errors or empty results — accidentally caching a failure makes a transient blip persistent.

Where to go next

Caches, replicas, and networks all fail. The mature response is to expect it. Next: designing for failure.