One node cannot absorb every row and write forever#
Sharding horizontally partitions rows so each node owns a subset. The shard key determines routing, locality, balance, cross-shard work, and eventually migration cost.
Hash, range, and directory placement#
| Strategy | Strength | Trade-off |
|---|---|---|
hash(user_id) % 4 | Can distribute varied keys | Range queries scatter; shard-count changes remap heavily |
| A–F, G–M, N–Z or time ranges | Range locality | Skew and sequential-write hotspots |
customer_123 → shard_7 | Flexible placement | Directory availability, consistency, and operation |
Choose the key from dominant access patterns#
- Prefer high cardinality and a distribution compatible with expected traffic.
- Co-locate data commonly queried or changed together.
- Avoid keys whose hottest values overwhelm one shard.
- Choose a stable key; changing it later means moving data and routing.
user_id may work for user-scoped records. country can skew when one country dominates. created_date can make today's range absorb every current write. Context decides.
Even distribution does not eliminate hot keys#
A celebrity account can overwhelm its owning shard even when millions of ordinary keys are balanced. Caching, replicas, key splitting, precomputation, or workload-specific fan-out may spread the hot operation.
A global query across user shards
If orders are partitioned by user_id, “all orders in the last hour” may query every shard and merge results. A dedicated time-oriented index or analytics pipeline may be more appropriate than making the transactional path perform a scatter-gather.
Four shards becoming eight is a migration#
Data must move while reads and writes continue. The router needs old/new ownership, migrations add load, and correctness must survive partial progress. Naïve modulo hashing makes many keys move; the next lesson shows why consistent hashing can help some changing-node workloads.
Atomic operations across shards also require more coordination. Cross-link later to Distributed Transactions and Two-Phase Commit.
Use these prompts to extend the design. The useful answer is a reasoned change, not a memorized sentence.
- Why this shard key?
- What becomes hot?
- How does a global query run?
- How do cross-shard changes behave?
- How do you reshard without stopping traffic?
Related practice problems
Use the manifest-backed prompts below to apply this lesson in a complete design.
- DynamoDB data-modeling best practicesAWS Documentation
- Cassandra partition keysApache Cassandra Documentation