Load Balancing

Clients need one stable entry, not a server inventory#

You add a second application server, then a third. Clients need a way to reach a healthy instance without learning which machines exist. A load balancer accepts traffic and selects a target according to health, routing policy, and available request information.

The workload should influence the algorithm#

Consistent hashing reduces remapping when targets change, but hot keys and uneven work still require explicit handling.

Health checks remove targets; draining protects in-flight work#

Periodic checks mark a backend unhealthy and restore it after recovery criteria are met. A shallow check can pass while a critical dependency is broken; a deep check can remove every backend during a shared dependency failure. Thresholds, timeouts, startup readiness, and connection draining shape the behavior.

Sticky sessions, L4, and L7#

Sticky routing sends a client back to the same target. It can simplify local session state, but backend failure loses affinity and uneven clients produce uneven load. Shared or external session state often makes application replicas easier to scale, though stickiness is not inherently wrong.

An L4 balancer routes from transport information such as IP and port. An L7 balancer understands HTTP information such as hostname, path, or headers, enabling rules like:

/api/payments/* → Payment Service
/api/users/*    → User Service

L7 adds application-aware flexibility and processing; L4 can operate with less application interpretation.

The balancing layer also needs redundancy#

If every request passes through one load-balancer process, it can become a single failure point. Production deployments use redundant instances and health/failover mechanisms, with DNS, virtual addresses, or managed infrastructure supplying the reachable entry point.