Loading Engineering Foundry…
Loading Engineering Foundry…
Concept directory
20 interview-ready lessons connect ML judgment to the production system around it.
Start in order if ML system design is new. If you already build ML systems, use the decision trigger to find the lesson that matches the part of your design you cannot yet defend.
Start with problem formulationDecide whether ML is justified before selecting a model family.
Foundation · What measured baseline gap, workload, or harm pattern would justify introducing ML?Build a layered metric map with task-specific quality and operational guardrails.
Foundation · Which user outcome or guardrail movement would make an offline gain unacceptable?Trace a product event into a trustworthy training example.
Foundation · Which missing event or quality observation would invalidate the training set?Define positive, negative, unknown, and mutable label states with a maturation policy.
Foundation · What delay, disagreement, or reversal rate would change the label policy?Choose temporal, entity, or group splits and perform point-in-time joins.
Foundation · Which deployment difference would make this split invalid?Turn raw signals into available, stable, and useful decision-time features.
Foundation · What incremental quality gain would justify this feature's latency and ownership cost?Explain what an embedding represents and how it moves through retrieval and rollout.
Intermediate · What recall gap or corpus property makes dense retrieval worth the lifecycle cost?Select weighting and negative sampling based on exposure and deployment prevalence.
Intermediate · What prevalence or hard-negative error would change the sampling strategy?Match the metric to the decision and expose errors hidden by an average.
Foundation · Which error type or slice result would block deployment despite an average gain?Choose thresholds from consequences and queue capacity rather than a default score.
Intermediate · What cost, prevalence, or reviewer capacity would move the threshold bands?Design a rerunnable pipeline with explicit publish gates and lineage.
Intermediate · What failure or audit question would the current lineage be unable to answer?Decide whether a feature store is earned and define its correctness contract.
Intermediate · What reuse, latency, or correctness cost makes a shared feature platform worthwhile?Distinguish batch scoring, streaming updates, precomputation, online inference, and hybrid paths.
Intermediate · What freshness or context requirement rules out precomputation?Allocate the full request budget and design saturation and fallback behavior.
Intermediate · Which p99, throughput, or cost observation would force a smaller model or different compute path?Separate artifact registration from deployment and define compatibility and rollback signals.
Intermediate · Which compatibility or guardrail result blocks the next traffic step?Know when an online experiment is appropriate and what can invalidate it.
Intermediate · What integrity or guardrail signal would stop the experiment before significance?Separate drift signals from root cause and product impact.
Intermediate · Which combination of signals distinguishes bad data from changed user behavior?Identify endogenous labels and choose exploration and retraining triggers deliberately.
Advanced · What evidence shows the current policy is narrowing the data it learns from?Design SLOs, timeouts, stale modes, fallbacks, load shedding, and recovery.
Intermediate · Which dependency failure requires a safe stop instead of a lower-quality fallback?Map concrete risks to preventive, detective, corrective, and human controls.
Cross-cutting · What harm, uncertainty, or reversibility limit requires human oversight or no automation?