The protective layer can fail#
A cache can be unavailable, slow, partitioned, cold, stale, memory-constrained, or overloaded by a hot key. Failure behavior depends on the data and source capacity; “just query the database” can turn a cache incident into a database outage.
A cache outage at 100,000 RPS
Normal DB load = 100,000 × 0.05 = 5,000 RPS
Uncontrolled bypass ≈ 100,000 RPS
Capacity shortfall ≈ 90,000 RPSBound fallback concurrency, load shed, degrade expensive features, use a safe local fallback, or serve permitted stale values. Recover gradually so cache repopulation does not create a second surge.
| Failure | Consequence | Design response |
|---|---|---|
| Cache unavailable / timeout | Latency and origin surge | Bounded bypass, degradation, or fail request |
| Mass expiration | Synchronized misses | TTL jitter and coalescing |
| Cold cache | High backend load | Prewarm and gradual traffic |
| Stale data | Product correctness error | Freshness classes and revalidation |
| Memory pressure | Useful values evicted early | Admission, sizing, and eviction telemetry |
| Hot key | One node saturates | Replicas, local copies, or split work |
Ask what is safer for this data#
- Product catalog: limited database bypass or bounded stale reads may be acceptable.
- Massive feed: uncontrolled bypass can collapse fan-out and storage dependencies.
- Authorization: serving stale or bypassed values may violate a security requirement.
The later Load Shedding and Circuit Breaker lessons go deeper into dependency protection.
- Designing only the hit path.
- Using a global hit rate that hides one expensive miss class.
- Recovering at full traffic and immediately expiring another large key set.
Use these prompts to extend the design. The useful answer is a reasoned change, not a memorized sentence.
- Can the database survive a full cold cache?
- Which responses may be stale?
- Which features should degrade first?
- How do you test cache-disabled operation?
Related practice problems
Use the manifest-backed prompts below to apply this lesson in a complete design.
- Caching challenges and strategiesAmazon Builders' Library
Reach this point to mark the lesson as read.