Kafka packages the concepts you already learned#
Kafka is a distributed retained event-log platform built from clusters, brokers, topics, partitions, producers, consumers, consumer groups, offsets, retention, and replication. It makes sense only after streams and partition-scoped ordering do.
Records remain according to retention or compaction policy rather than disappearing because one group consumed them. Independent groups can read the same history and track their own positions.
Estimate raw event ingress
20,000 × 2 KB ≈ 40 MB/sec
40 MB/sec × 86,400 ≈ 3.46 TB/dayThis excludes replication, indexes/metadata, compression, and protocol overhead. Partition count depends on measured producer and consumer throughput, hardware, record size, replication, latency, and headroom - not a universal MB/sec constant.
- Saying “use Kafka” without a replay, retention, consumer, or ordering requirement.
- Claiming global topic order.
- Assuming retained means forever.
Related practice problems
Use the manifest-backed prompts below to apply this lesson in a complete design.
- Apache Kafka introductionApache Kafka Documentation
- Kafka design and delivery semanticsApache Kafka Documentation