Core concept · Foundation

Labels, Ground Truth & Label Delay

Define positive, negative, unknown, and mutable label states with a maturation policy.

Content reviewed September 2026Read to the end

What you will learn

Define positive, negative, unknown, and mutable label states with a maturation policy.

Decision trigger

What delay, disagreement, or reversal rate would change the label policy?

Interview impact

Label design changes dataset windows, retraining cadence, evaluation, review, and appeals.

Mutable label timeline

1Exposure
2Pending
3Mature outcome
4Dispute
5Corrected label

Mental model

A label is an operational definition of evidence, not objective truth.

A purchase that occurs a day after an ad click is prematurely labeled negative at training time.

How it works

Define the observation window and states such as pending, positive, confirmed negative, reversed, and disputed.
Record label source, policy version, reviewer context, and correction history.

Product consequences

Premature negatives bias delayed-conversion and long-horizon systems.

Operational consequences

Mutable labels require recomputation, lineage, adjudication, and backfill controls.

Alternatives

Use weak supervision when verified outcomes are scarce, and measure disagreement.
Use positive-unlabeled framing when non-events are not reliable negatives.

Failure modes

Reviewer decisions are treated as infallible ground truth.
Label policy changes without versioning historical examples.

Worked example

A chargeback label remains pending through the dispute window.
Reversals update the label record and future datasets without silently rewriting prior experiments.

Interviewer probes

When does a negative become trustworthy?
How do corrections reach old datasets?

Expected depth by level

Entry: Define a clear positive and negative.
Mid-level: Handle maturation, noise, and correction.
Senior+: Design label operations, adjudication, auditability, and policy migrations.
Role emphasis: Applied Scientist owns target validity; MLE owns operational label state.

System Design prerequisites

Event-time processing

Sources and review notes

These sources support the technical claims on this page. Product-specific examples remain Engineering Foundry teaching scenarios.

Rules of Machine LearningGoogle for Developers · first-party engineering/science · reviewed 2026-09-04
Baselines, instrumentation, launch sequencing, and production iteration guidance.
Hidden Technical Debt in Machine Learning SystemsGoogle Research · original paper · reviewed 2026-09-04
Feedback loops, data dependencies, configuration debt, and system-level ML risks.

Reach this point to mark the lesson as read.

Finish this lesson

Record completion when you have worked through the material. Completion is learning progress, not evidence of mastery.

Loading completion…

Previous: Data Collection, Logging & Data Quality

Loading Engineering Foundry…

Sources and review notes

These sources support the technical claims on this page. Product-specific examples remain Engineering Foundry teaching scenarios.

Rules of Machine LearningGoogle for Developers · first-party engineering/science · reviewed 2026-09-04

Baselines, instrumentation, launch sequencing, and production iteration guidance.

Hidden Technical Debt in Machine Learning SystemsGoogle Research · original paper · reviewed 2026-09-04

Feedback loops, data dependencies, configuration debt, and system-level ML risks.

Decision trigger

Interview impact

Mental model

How it works

Product consequences

Operational consequences

Alternatives

Failure modes

Worked example

Interviewer probes

Expected depth by level

System Design prerequisites

Practice this concept

Sources and review notes

Finish this lesson

Decision trigger

Interview impact

Mental model

How it works

Product consequences

Operational consequences

Alternatives

Failure modes

Worked example

Interviewer probes

Expected depth by level

System Design prerequisites

Practice this concept

Sources and review notes

Finish this lesson