Share a working set across application instances#
A distributed cache is useful when one machine cannot hold or serve the working set, or many application instances need shared cached state. It adds capacity, throughput, and redundancy by partitioning and often replicating data across nodes.
Distribution creates a new system to operate#
- Partitioning: choose which node owns a key.
- Replication: decide how copies acknowledge and fail over.
- Rebalancing: move ownership when nodes join or leave.
- Network latency: every shared-cache access is a remote call.
- Hot keys: one popular key can still overload one owner.
- Partial failure: clients need bounded timeouts and recovery behavior.
The Consistent Hashing lesson explains how a changing node set can limit key movement. It does not guarantee uniform traffic or eliminate rebalancing.
| Property | Local cache | Distributed cache |
|---|---|---|
| Read latency | Fastest; no network hop | Includes network and cluster routing |
| State | Per application instance | Shared across clients |
| Capacity | Bounded by one process | Partitioned across nodes |
| Invalidation | Hard across many copies | Centralized policy, still distributed internally |
| Failure | Lost with process | Cluster is a shared dependency |
A hybrid can use a tiny local L1 in front of a shared L2 and then the database. That can protect a hot distributed key but creates another stale copy and additional invalidation rules.
- Calling a distributed cache “highly available” without a replication and failover design.
- Assuming even key distribution means even request distribution.
- Ignoring client behavior during rebalancing or partial node failure.
Use these prompts to extend the design. The useful answer is a reasoned change, not a memorized sentence.
- How are keys assigned when a node is added?
- What happens when one node fails?
- What if one key receives 100,000 RPS?
- How does the database survive cluster-wide failure?
Related practice problems
Use the manifest-backed prompts below to apply this lesson in a complete design.
- Caching challenges and strategiesAmazon Builders' Library
- Redis replicationRedis Documentation