Caching is a reuse decision before it is a product choice#
A cache keeps a reusable copy of data or completed work closer to the next consumer. It helps when many requests repeat an expensive database read, computation, remote call, or geographic transfer. A hit returns a usable copy; a miss continues to the source and may populate the cache.
Caching trades memory and potential staleness for lower latency and less backend work.
The trade is worthwhile only when reuse is likely. Read-heavy, slow-changing data and hot queries are common candidates. Low-reuse, write-heavy, correctness-critical, huge, or cheap-to-recompute values may produce little benefit while adding invalidation and failure paths.
A hit rate is an origin-load assumption#
Protect a database at 50,000 reads per second
DB RPS = total RPS × (1 − hit rate)
= 50,000 × 0.10
= 5,000 RPSThe cache removes about 45,000 repeated reads each second under these assumptions. If it disappears, the database may receive the full 50,000 RPS. The design must prove that fallback is bounded or deliberately degraded.
When does caching fit?#
| Likely to help | Likely to disappoint or harm |
|---|---|
| Popular, read-heavy objects | Personalized one-off results |
| Slow or costly computations | Cheap backends with little repeated work |
| Remote calls and static or semi-static content | Strict freshness with no safe stale window |
| Hot database queries with bounded values | Very large, low-reuse values or write-heavy state |
Edge response caching belongs in the CDN lesson. Here the same reasoning applies to application data: identify reuse, key identity, freshness, capacity, and source behavior.
- Saying “add Redis” without naming the data, key, TTL, invalidation, or outage behavior.
- Assuming a constant or 100% hit rate.
- Caching balances, permissions, or inventory without a staleness policy.
- Caching everything even when low-reuse entries evict the useful working set.
Use these prompts to extend the design. The useful answer is a reasoned change, not a memorized sentence.
- What data would you cache, and what is the cache key?
- How stale may it become?
- Can the source survive a cold cache?
- Which metrics prove the cache is helping?
Related practice problems
Use the manifest-backed prompts below to apply this lesson in a complete design.
- Caching challenges and strategiesAmazon Builders' Library
- RFC 9111: HTTP CachingIETF / RFC Editor