Core concept · Intermediate

Class Imbalance, Sampling & Negative Examples

Select weighting and negative sampling based on exposure and deployment prevalence.

Content reviewed September 2026Read to the end

What you will learn

Select weighting and negative sampling based on exposure and deployment prevalence.

Decision trigger

What prevalence or hard-negative error would change the sampling strategy?

Interview impact

Negative construction changes what the model learns and what offline metrics mean.

Sampling changes prevalence

1Natural traffic
2Eligibility
3Sample policy
4Training mix
5Corrected evaluation

Mental model

Sampling changes the data distribution; evaluation and calibration must know how.

A fraud detector reports 99.9% accuracy by predicting every rare positive as negative.

How it works

Measure natural prevalence and distinguish exposed non-events from unknown candidates.
Choose weighting, under/oversampling, and hard negatives; correct evaluation for the sampled distribution.

Product consequences

Poor negative design produces irrelevant recommendations or missed rare harms.

Operational consequences

Sampling policies must be versioned with datasets and monitored as inventory changes.

Alternatives

Use anomaly detection only when labeled positives are genuinely unavailable and validate operational usefulness.
Use rules for rare high-harm signatures with insufficient learning evidence.

Failure modes

Offline prevalence differs from production with no correction.
Unexposed items are labeled disliked.

Worked example

Fraud training weights rare positives and evaluates on natural prevalence.
Recommendation samples from eligible impressions plus hard near-misses, while treating unseen items as unknown.

Interviewer probes

Where did negatives come from?
Does evaluation preserve production prevalence?

Expected depth by level

Entry: Recognize accuracy failure under imbalance.
Mid-level: Design exposure-aware negatives and evaluation.
Senior+: Own sampling correction, exploration, reviewer capacity, and adversarial adaptation.
Role emphasis: Applied Scientist deepens estimator effects; MLE owns pipeline versioning.

System Design prerequisites

Capacity estimation

Sources and review notes

These sources support the technical claims on this page. Product-specific examples remain Engineering Foundry teaching scenarios.

Classification: accuracy, recall, precision, and related metricsGoogle for Developers · official documentation · reviewed 2026-09-04
Task-sensitive classification metrics and the limitations of accuracy on imbalanced data.
Deep Neural Networks for YouTube RecommendationsGoogle Research · original paper · reviewed 2026-09-04
Historical two-stage candidate generation and ranking example; not a claim about a current production system.

Reach this point to mark the lesson as read.

Finish this lesson

Record completion when you have worked through the material. Completion is learning progress, not evidence of mastery.

Loading completion…

Previous: Embeddings & Learned Representations

Loading Engineering Foundry…

Sources and review notes

These sources support the technical claims on this page. Product-specific examples remain Engineering Foundry teaching scenarios.

Classification: accuracy, recall, precision, and related metricsGoogle for Developers · official documentation · reviewed 2026-09-04

Task-sensitive classification metrics and the limitations of accuracy on imbalanced data.

Deep Neural Networks for YouTube RecommendationsGoogle Research · original paper · reviewed 2026-09-04

Historical two-stage candidate generation and ranking example; not a claim about a current production system.

Decision trigger

Interview impact

Mental model

How it works

Product consequences

Operational consequences

Alternatives

Failure modes

Worked example

Interviewer probes

Expected depth by level

System Design prerequisites

Practice this concept

Sources and review notes

Finish this lesson

Decision trigger

Interview impact

Mental model

How it works

Product consequences

Operational consequences

Alternatives

Failure modes

Worked example

Interviewer probes

Expected depth by level

System Design prerequisites

Practice this concept

Sources and review notes

Finish this lesson