A miss on a hot key is not necessarily one origin request#
When a popular entry expires, thousands of concurrent readers can observe the same miss and repeat the same source work. The cache normally protects the database; synchronized misses temporarily remove that protection.
A profile key expires under 20,000 RPS
20,000 × (1 − 0.99) = 200 DB queries/secIf the key expires and all concurrent reads query the source, load can jump toward 20,000 QPS - twenty times the illustrative capacity.
One expired key, two very different outcomes
Expire a hot key, then enable request coalescing to compare origin load.
Accessible request-state summary
| Cache state | Coordination | DB queries |
|---|---|---|
| Fresh | Not needed | 0 |
| Expired | None | 10,000 |
| Expired | Single flight | 1 |
Coordinate and spread refresh work#
- Request coalescing / single flight: share one in-progress load among concurrent readers.
- Per-key locking: elect one refresher, with bounded waits and safe lock expiry.
- TTL jitter: keep large key sets from expiring together.
- Refresh before expiry: proactively refresh known hot keys.
- Stale-while-revalidate: serve a permitted old value while one worker refreshes.
Use these prompts to extend the design. The useful answer is a reasoned change, not a memorized sentence.
- What happens when the refresh worker dies?
- How long may waiters block?
- Can stale data be served during refresh?
- How do you avoid synchronizing many different keys?
Related practice problems
Use the manifest-backed prompts below to apply this lesson in a complete design.
- Caching challenges and strategiesAmazon Builders' Library
Reach this point to mark the lesson as read.