Start with the decision#
A read-heavy system should optimize the few dominant access paths without turning every response into an untraceable stack of stale copies. Start with the read distribution, not the total request count.
Mechanism and state flow#
- Rank reads by frequency, latency objective, payload, and freshness.
- Precompute stable projections and place them near their consumers.
- Use edge, application, and database caches only where each layer has a distinct ownership and invalidation contract.
- Replicate or partition the remaining misses, preserving a route for authoritative reads.
- Model a full miss storm and cap regeneration concurrency before launch.
Concrete example#
A public news homepage
Anonymous headlines are edge cached, article cards use a versioned projection, and personalized badges are fetched separately. Editors preview from the authoritative store rather than the public cache.
Architecture consequence: Compose data with different freshness policies instead of weakening the strictest field or overloading the origin for the loosest one.
Trade-offs#
| Option | Choose when | Cost or limit |
|---|---|---|
| Precomputed projection | A common query shape is expensive but changes predictably. | Projection lag and rebuild operations must be visible. |
| Replica query | Query flexibility matters more than immediate freshness. | Lag and replica capacity still constrain the path. |
| Origin query | The decision requires current authoritative state. | Expensive fan-in cannot scale by caching unrelated fields. |
Failure diagnosis#
- Failure
- A cache expiry synchronizes millions of requests on one expensive query.
- Impact
- The database saturates and healthy cached paths fail behind the same shared pool.
- Detection
- Hit ratio drops while concurrent regeneration, connection wait, and origin latency spike together.
- Mitigation
- Use jittered expiry, request coalescing, stale-while-revalidate where safe, and isolated capacity for regeneration.
- Trade-off
- Serving stale data extends the period before every reader sees an update.
Design checkpoint#
Work the scenario before revealing the checklist
- Draw the top three read paths and their cache keys.
- State freshness and invalidation separately for every response field.
- Estimate origin QPS at normal hit rate and at zero hit rate.
- Choose the safe behavior when the projection builder is behind.
Use these prompts to extend the design. The useful answer is a reasoned change, not a memorized sentence.
- What is the read access skew?
- Which layer owns invalidation?
- How do privileged and public views avoid sharing a cache entry?
- What protects the origin during a cold start?
Related practice problems
Use the manifest-backed prompts below to apply this lesson in a complete design.
- Caching challenges and strategiesAmazon Builders' Library
- PostgreSQL warm standby and replicationPostgreSQL Documentation
Sources reviewed 2026-09-04. Service limits and product-specific guarantees still require current verification.