Mental model#
A log is a timestamped event record, not a stream of prose. Its value comes from stable fields that let an operator filter by service, severity, operation, outcome, trace, and safe business key without exposing secrets or whole payloads.
Mechanism and state flow#
- Define a small event schema with time, severity, body, resource, attributes, and correlation context.
- Emit one event for a meaningful state transition or failure boundary rather than every line of code.
- Redact credentials, tokens, personal data, and unrestricted request bodies before export.
- Buffer and ship asynchronously; index only fields needed for bounded operational queries.
- Apply retention and access by sensitivity, then test that deletion and incident access work.
Concrete example#
A duplicate payment request
The payment handler logs operation name, safe payment reference, idempotency outcome, dependency status, trace ID, and duration. It does not log the bearer token, card data, or complete request body.
Architecture consequence: Use structured fields for the retry decision and join to a trace; keep financial or personal data outside the general log pipeline.
Scaling and operational trade-offs#
| Option | Choose when | Cost or limit |
|---|---|---|
| Structured events | Operators need reliable filtering and aggregation. | Schemas and field governance require maintenance. |
| Full-text messages | Human context is useful during early development. | Parsing is brittle and accidental sensitive data is harder to control. |
| Long retention | Audit or slow incident discovery requires it. | Storage, access risk, and deletion obligations grow. |
Failure diagnosis#
- Failure
- A release logs entire authorization headers on errors.
- Impact
- Credentials are copied into a widely accessible, long-lived system.
- Detection
- Secret-scanning rules and sampled pipeline inspection flag token-shaped values.
- Mitigation
- Redact at instrumentation and collector boundaries, revoke exposed credentials, restrict the dataset, and shorten retention.
- Trade-off
- Aggressive redaction can remove debugging detail, so preserve safe identifiers and explicit allowlisted fields.
Checklist exercise#
Work the scenario before revealing your answer
- Rewrite an unstructured timeout message as stable fields.
- Classify each field as index, retained body, redacted, or prohibited.
- Choose sampling and retention separately for normal events and security/audit events.
Use these prompts to extend the design. The useful answer is a reasoned change, not a memorized sentence.
- What does this log let an operator decide?
- Which fields may contain secrets or personal data?
- How do schema changes and timestamp skew affect queries?
- What happens when export buffers fill?
Related practice problems
Use the manifest-backed prompts below to apply this lesson in a complete design.
- OpenTelemetry logging specificationOpenTelemetry
- Context propagationOpenTelemetry
Sources reviewed 2026-09-04. Product-specific controls and threat assumptions still require current verification.