Core System Properties

Use properties to test a design#

Your service handles 2,000 requests per second today. The interviewer asks what happens at 200,000, then removes a server, slows the database, and adds users on another continent. Core system properties give you precise questions for each change.

Scalability: can goals survive workload growth?#

Scalability is not simply “handles more traffic.” Ask which workload grows, which resource saturates, and whether latency, correctness, and availability still meet the agreed goals.

Vertical scaling

Add CPU, memory, storage, or network capacity to one machine. It is operationally simple and often the correct early step, but each machine has a ceiling, larger instances can be expensive, and the failure domain remains concentrated.

Horizontal scaling

Add machines and distribute work. This supports incremental growth and can distribute failures, but creates load-balancing, coordination, partitioning, deployment, and observability work.

Latency and throughput answer different questions#

Latency measures how long one operation takes. Throughput measures how much work completes per unit of time. A service can process 100K requests per second while an individual request still takes 800 ms. High throughput does not imply low latency.

Interviewers care about throughput for capacity and latency for user-visible behavior. Batching may improve throughput while making an individual item wait longer; extra parallelism may reduce one bottleneck while increasing contention elsewhere.

Percentiles reveal the tail#

p50 = 80 ms
p95 = 250 ms
p99 = 1.2 sec

About half of requests finish at or below 80 ms, 95% at or below 250 ms, and 99% at or below 1.2 seconds. The remaining 1% are slower than that p99 value. These are example measurements, not targets.

An average can look healthy while a small but important group waits much longer. Tail latency matters because a page may depend on several services and because a 1% problem affects many users at large request volume.

Availability: can the operation be served when needed?#

Availability should be defined for a concrete operation: successful redirects, readable conversations, or accepted payments. At 99.9% time-based availability, the unavailable budget is roughly 8.76 hours per 365-day year. That approximation is context, not a table to memorize.

Redundant instances, health checks, failover, multiple zones, and isolation from failing dependencies can improve availability. The design still needs a behavior for partial failure: reject, retry, serve stale data, queue work, or degrade a feature.

Reliability and correctness: successful responses can still be wrong#

A payment API may return a successful response while charging twice. It was available for that request, but it did not behave reliably or preserve the required invariant. In an interview, say which outcomes must be correct and how retries, duplicate requests, and partial completion are handled.

Durability: what survives after acknowledgement?#

An uploaded photo, transaction record, or message history is durable when the system can preserve the acknowledged write across the failures in scope. Persistent storage, replication, acknowledgement rules, and backups all contribute, but they protect against different failures and restore at different speeds.

Fault tolerance: design expected failures into the path#

Fault tolerance does not mean nothing fails. It means the system can continue, recover, or degrade predictably when components fail. Redundancy, bounded retries, replication, failover, isolation, and graceful degradation are tools; each requires a stated failure model.

Bottlenecks: find the constrained resource#

API capacity:       100K RPS
Cache capacity:     200K RPS
Database capacity:    8K RPS

Look beyond CPU: memory, network bandwidth, storage IOPS, locks, hot partitions, connection limits, and downstream services can all constrain the system.

Redundancy buys options, not free reliability#

Multiple instances can preserve service through one instance failure, but they introduce routing, health checking, replication, failover, consistency, and cost questions. Redundant components that share one power, network, configuration, or data dependency may still fail together.

A compact property-to-technique map#

These are starting points, not laws. The right technique depends on the operation, workload, failure model, and budget.