Finding the component group
Macro-F1 balances performance across component groups. Higher is better; 1 is a perfect score.
The AI models outperform the free text classifier on this sample. The comparison model ran on fewer cases.
From complaint
to next test
Field Return Triage · independent project
A returned part can pass the standard test and still have a fault. I built a tool that finds failure clues in the complaint, checks the exact words behind them, and suggests what a technician could test next.
Uses public vehicle complaints as stand-ins for return reports. The suggested tests are examples to adapt to a lab.
The AI suggests. Code checks. People decide. No result closes, scraps or returns a part.
What I built
The complaint-to-card pipeline, verbatim quote checks, safety rules, similar-case search, a configurable test rulebook, and a reproducible evaluation.
Follow the evidence
Choose a recorded case. Compare the original words with the suggested test, then try the review controls.
Recorded examples, not live AI. These controls make no requests and save no decisions.
How these examples were selected
Useful, with limits
The results describe this evaluation sample. They do not establish performance on real returned parts.
Safety flags need context
Primary model plus hard rules, compared with the coded crash, fire and injury fields.
The evaluation deliberately oversamples safety cases. High recall comes with many extra cases for a person to read.
Full scores, confidence intervals and free baselines
Brackets show 95% bootstrap confidence intervals: the uncertainty around each estimate. “Unclear” counts as wrong for component-group scoring. The comparison model stopped early at the budget cap, so these are not equal-size model trials.
The free baselines
Model selection and reproducibility
The page does not pick a model. The project locks its choices to a versioned evaluation card, with a pinned source commit.
Do complaint spikes help?
A separate rule watches weekly complaint counts. The useful question is whether its alerts precede recalls more often than random weeks do.
Inspect the weekly series and backtest method
Weekly complaintsAlert Recall Recall window
Exact weekly values
| Week starting | Complaints | Event |
|---|
Before taking this further
- Public data, different setting
- US vehicle complaints stand in for return reports. They do not contain a lab’s test history or represent one supplier’s parts.
- Labels are imperfect
- Component and safety fields come from a public coding process, not an engineer’s finding. Scores inherit that uncertainty.
- Test suggestions are examples
- A lab needs its own test catalogue, inspection points and escalation rules. A suggested test is not a confirmed diagnosis.
- Agreement is not accuracy
- Conditions such as moisture or vibration have no ground-truth labels here. Their score measures agreement with a fixed keyword rule.
This is a measured portfolio project. It has no production queue, authentication or validated deployment in a returns lab.
Check the evidence yourself
Download evidenceThe same committed data that renders this page, including the example cards and uncertainty intervals.
A traceable next step
Which returned parts need a closer look, and which test should run first? The flow keeps the original complaint attached to every suggestion.
- AI extraction
Find the clues
Read the complaint for symptoms, conditions and safety indications. Return a quote for each clue.
- Code
Check the words
Drop any clue whose quote does not appear in the original complaint.
- Hard rules
Put safety first
Safety goes to priority 1 with human review. Unclear cases go to human reading.
- Search + rulebook
Suggest the test
Find similar past cases. Map the checked clues to tests, inspections and questions.
- Technician
Make the decision
Accept the suggestion or change it with a reason. The real pipeline logs this feedback.
Implementation, replay and adaptation
Repeat the evaluation
The Python command-line pipeline replays committed model responses. Its tests run with network access blocked. New text requires an explicitly configured model run and budget.
make page
make facts
make testAdapt the domain knowledge
The component taxonomy and test rulebook are configurable. A lab would supply its own parts, complaint records and actual tests. The code still checks quotes, enforces safety rules and requires a person’s decision.
The separate trend path uses weekly counts and a statistical spike rule, without calling an AI model.