ML System Design · Engineering Foundry challenge level: Intermediate

Support Ticket Routing with Abstention

Design an ML-assisted router that assigns support queues when confident and safely abstains when it is not.

All challengesSuggested practice time: 75 minutes

Original prompt

What you are building

Design an ML system that recommends a support queue for each incoming ticket while preserving a manual triage path. The system should optimize time to appropriate ownership without silently misrouting rare or high-risk requests.

Engineering Foundry originalNo company association

Deliverables

Problem framing, prediction target, and abstention policy.
Offline, online, and guardrail metrics.
Training data and label construction.
Baseline and model/system approach.
Serving and human-in-the-loop workflow.
Monitoring, feedback, and failure analysis.

Constraints

Manual triage must remain available.
Sensitive ticket text needs access controls and retention limits.
The model must handle new queues and changing taxonomy.
A recommendation is not an automatic resolution decision.

Success criteria

Labels account for transfers and noisy historical choices.
Abstention and human review are first-class behavior.
Metrics distinguish routing speed from routing correctness.
Drift, taxonomy change, and subgroup failures are monitored.

Suggested workflow

Define the decision and the cost of each error.
Create a label policy from noisy assignment history.
Set a baseline and abstention threshold strategy.
Design online serving and manual fallback.
Close the loop with delayed outcome and drift monitoring.

Attempt before reveal

Build your deliverable independently.

Use the prompt, constraints, and workflow first. Guidance offers considerations and questions - not a canonical solution.

Reveal guidanceDirections and questions to test

Questions to test

The first assigned queue may not be the best label when a ticket is transferred.
A rules or retrieval baseline can expose whether a learned model adds value.
Confidence calibration matters when the product action includes abstention.

Reveal common mistakesFailure patterns worth checking

Optimizing only top-one accuracy.
Training directly on every historical assignment.
Forcing a model prediction for every ticket.
Ignoring queue taxonomy changes and new-product cold starts.

Reveal stretch goalsOptional ways to extend the deliverable

Recommend both queue and required specialist skill.
Design an active-learning queue for high-value uncertain examples.

Optional solution worksheet

Prepare your approach for discussion.

Capture your reasoning, then copy a plain-text summary you control.

Session only - not saved.Refreshing or leaving this page clears every worksheet field and self-review choice. Nothing is uploaded or officially submitted.

Approach summary

Key decision

Important tradeoff

Failure mode or edge case

Validation or testing plan

Reflection

Optional solution URLThis link remains in the current browser session and is included only if you copy the summary.

Nothing is sent when you copy.

Optional community discussion

Compare reasoning, not rankings.

Discuss this challenge in the Engineering Foundry community if you choose. Share your own work, explain assumptions, critique ideas rather than people, and remove confidential or proprietary material.

Discuss this challenge

Loading Engineering Foundry…

ML System Design · Engineering Foundry challenge level: Intermediate

Support Ticket Routing with Abstention

Design an ML-assisted router that assigns support queues when confident and safely abstains when it is not.

All challengesSuggested practice time: 75 minutes

Original prompt

What you are building

Engineering Foundry originalNo company association

Deliverables

Problem framing, prediction target, and abstention policy.
Offline, online, and guardrail metrics.
Training data and label construction.
Baseline and model/system approach.
Serving and human-in-the-loop workflow.
Monitoring, feedback, and failure analysis.

Constraints

Manual triage must remain available.
Sensitive ticket text needs access controls and retention limits.
The model must handle new queues and changing taxonomy.
A recommendation is not an automatic resolution decision.

Success criteria

Labels account for transfers and noisy historical choices.
Abstention and human review are first-class behavior.
Metrics distinguish routing speed from routing correctness.
Drift, taxonomy change, and subgroup failures are monitored.

Suggested workflow

Define the decision and the cost of each error.
Create a label policy from noisy assignment history.
Set a baseline and abstention threshold strategy.
Design online serving and manual fallback.
Close the loop with delayed outcome and drift monitoring.

Attempt before reveal

Build your deliverable independently.

Use the prompt, constraints, and workflow first. Guidance offers considerations and questions - not a canonical solution.

Reveal guidanceDirections and questions to test

Questions to test

The first assigned queue may not be the best label when a ticket is transferred.
A rules or retrieval baseline can expose whether a learned model adds value.
Confidence calibration matters when the product action includes abstention.

Reveal common mistakesFailure patterns worth checking

Optimizing only top-one accuracy.
Training directly on every historical assignment.
Forcing a model prediction for every ticket.
Ignoring queue taxonomy changes and new-product cold starts.

Reveal stretch goalsOptional ways to extend the deliverable

Recommend both queue and required specialist skill.
Design an active-learning queue for high-value uncertain examples.

Optional solution worksheet

Prepare your approach for discussion.

Capture your reasoning, then copy a plain-text summary you control.

Session only - not saved.Refreshing or leaving this page clears every worksheet field and self-review choice. Nothing is uploaded or officially submitted.

Approach summary

Key decision

Important tradeoff

Failure mode or edge case

Validation or testing plan

Reflection

Optional solution URLThis link remains in the current browser session and is included only if you copy the summary.

Nothing is sent when you copy.

Optional community discussion

Compare reasoning, not rankings.

Discuss this challenge

Support Ticket Routing with Abstention

What you are building

Deliverables

Constraints

Success criteria

Suggested workflow

Build your deliverable independently.

Questions to test

ML System Design deliverable self-review

Prepare your approach for discussion.

Compare reasoning, not rankings.

Support Ticket Routing with Abstention

What you are building

Deliverables

Constraints

Success criteria

Suggested workflow

Build your deliverable independently.

Questions to test

ML System Design deliverable self-review

Prepare your approach for discussion.

Compare reasoning, not rankings.