Predictive Maintenance Pilot: 27% Precision and a Clock That Drifts Eight Minutes a Day
A plant installed condition monitoring on 3 of its 12 injection molding machines: 3-axis vibration at 25.6 kHz, hydraulic pressure as a proxy for screw torque at 1 Hz, and barrel heater current. Over six months those machines had 15 unplanned failures. The system raised 41 alerts: 11 were confirmed as developing failures and 30 were false. It missed 4 failures entirely.
Every alert costs a technician 45 minutes to inspect, and about one in four false alerts ends in a 2-hour unnecessary shutdown. An average unplanned failure means 4.2 hours of downtime at $3,800/h plus $2,200 of scrap. A planned repair takes 2 hours of downtime plus $900 in parts. The data gateway clock drifts up to 8 minutes per day and is corrected manually once a week; maintenance logs are free text; and 4 of the 15 failures were mold cooling problems on circuits that are not instrumented. Leadership wants to scale to all 12 machines next quarter.
1. Is the pilot economically justified as it stands? Compute the break-even precision and compare it with what the system actually delivered.
Read the worked answer
Cost of a miss. Unplanned failure = 4.2 h × $3,800 + $2,200 = $15,960 + $2,200 = $18,160.
Cost of a correct catch. Planned repair = 2 h × $3,800 + $900 = $7,600 + $900 = $8,500.
Value of a true positive. $18,160 − $8,500 = $9,660.
Cost of a false positive. 45 min inspection ≈ 0.75 h × $90 loaded labor ≈ $68, plus a 25% chance of the 2-hour shutdown: 0.25 × $7,600 = $1,900. Total ≈ $1,968.
Break-even precision. An alert is worth acting on when p × $9,660 > (1 − p) × $1,968, giving p* = 1,968 / (1,968 + 9,660) ≈ 16.9%. Below that, alerts cost more than they save.
Actual performance. Precision = 11/41 = 26.8%, recall = 11/15 = 73%. Precision is above the 16.9% break-even, so the pilot is economically positive - this is the key judgment. A 27% precision sounds poor if you benchmark against a laboratory classifier, but the asymmetry between a $9,660 save and a $1,968 nuisance is what matters.
Net value. 11 × $9,660 − 30 × $1,968 = $106,260 − $59,040 = +$47,220 over six months on three machines, before the 4 misses, which cost 4 × $18,160 = $72,640 in residual failures the system never had a chance to catch.
Conclusion. Justified, but the case for scaling should be built on the missed-failure coverage gap, not on the precision number alone. Do not set an arbitrary 80% precision target; it would trade away recall and lose money.
2. The 8-minute daily clock drift and free-text logs are not IT housekeeping. Explain how each corrupts the model, and identify the single largest risk to scaling.
Read the worked answer
Clock drift. Vibration features are only meaningful when they are aligned to the machine cycle - injection, hold, plasticizing, ejection. With an 8-minute-per-day drift corrected weekly, the offset wanders by up to roughly 56 minutes across a week. The result is that vibration snapshots get labelled with the wrong process phase, so identical signals appear under different operating states. The model learns phase-dependent noise as if it were failure signature, or misses a signature because it is smeared across states. Worse, when a maintenance event is joined to sensor data, the join is off by minutes, so the label may attach to the wrong shot entirely. Manual correction makes it nondeterministic: the same week reprocessed twice yields different training sets. Fix: NTP/PTP time synchronization on the gateway and PLC with drift monitoring, and store the correction history - never retro-correct silently.
Free-text logs. Class labels come from these records. 'Spindle noisy,' 'hyd. hiccup,' and 'PM done' are not failure modes. Without a controlled taxonomy - component, failure mode, detection time, repair action - supervised training has no reliable target, and precision/recall computed against them are themselves uncertain. Standardize the maintenance entry into structured fields.
Largest risk to scaling: coverage, not model quality. Four of 15 failures were mold cooling, which is not instrumented. No algorithm can detect what is not measured. Scaling the same sensor kit to 12 machines reproduces a 27% blind spot at 4× the cost. The right next step is failure-mode analysis per machine type, then sensor selection - add mold coolant flow and temperature before adding machines.
3. Define the acceptance criteria and rollout plan for the 12-machine scale-up.
Read the worked answer
Before scaling (4–8 weeks).
- Fix time synchronization and validate end-to-end latency from sensor event to database timestamp.
- Convert maintenance records to a structured taxonomy covering component, failure mode, detection, and action; back-label the existing six months.
- Instrument mold cooling (flow, supply/return temperature) on the three pilot machines and continue collecting until at least 10 cooling-related events are captured.
- Re-derive the alert threshold from the cost ratio rather than reusing a default: the operating point should maximize expected value, i.e. alert when p(failure|signal) × $9,660 > $1,968, which is the 16.9% break-even computed above.
Shadow-mode rollout. Bring the remaining nine machines online in two waves, sensors first, with alerts suppressed. Run 8–12 weeks and score against confirmed failures.
Acceptance criteria for enabling alerts.
- Precision ≥ 20% (comfortably above break-even, with margin for estimate error on small samples).
- Recall ≥ 70% on instrumented failure modes, measured on the full failure population, not just on alerts raised.
- Median alert-to-acknowledgement ≤ 30 minutes and ≤ 5% of alerts unacknowledged.
- Documented false-alert cost per machine per month, so finance can see the nuisance cost separately from the savings.
- A named owner for retraining and drift monitoring, with a defined trigger (e.g. precision falling below 15% over a rolling quarter).
Governance. Report precision and recall to the plant monthly alongside downtime. If a wave misses criteria, expand sensor coverage or narrow the claimed failure-mode scope - do not lower the bar to keep the rollout date.