Digital Twin Drift on the Gear Hobbing Line
A gear plant deployed a digital twin of six hobbing machines to predict tool wear from spindle current and vibration. During the first 90 days the model achieved 0.91 recall on confirmed tool breaks. In the last 30 days, false alarms rose 22% and two tool breaks were missed. The vibration sensor is mounted with a magnet; coolant chemistry changed two months ago; and the incoming material lot hardness is 8% higher than the commissioning baseline. The maintenance planner wants to retrain immediately on the last 30 days of data. The data scientist notes that only 14 confirmed tool breaks occurred in that period and that nine were labeled after the fact from maintenance notes.
1. Diagnose the likely causes of the rising false alarms and missed breaks. Separate installation effects, process changes, and model or data effects.
Read the worked answer
Do not start with retraining. The symptoms are consistent with several causes acting together.
Installation effects: a magnet-mounted vibration sensor can shift, lose coupling, or pick up resonant modes that change with mounting torque and temperature. Coolant chemistry can change damping and the vibration signature. Process changes: an 8% hardness increase raises cutting forces and spindle current, so the relationship learned at commissioning no longer holds. Model and data effects: the alarm threshold may be fixed while the feature distribution has shifted; the pipeline may have time-alignment errors; and the label set is weak because nine of fourteen breaks were reconstructed after the fact.
Evidence to collect includes sensor mounting torque and spectral baseline, a controlled cut with a known tool, feature distributions by machine and material lot, time alignment between current and vibration streams, and a review of each confirmed break to see whether the model saw precursors. Treat 'false alarm' and 'missed break' as separate error modes; they may have different causes.
2. The maintenance planner wants to retrain on the last 30 days. Give a go/no-go recommendation and a validation plan.
Read the worked answer
No-go for immediate retraining on 30 days. Fourteen events, with nine labeled retrospectively, cannot support a stable model and will encourage overfitting to the new material lot and coolant condition. Retraining without fixing the sensor or process shift may also bake in a bad baseline.
First, verify and if necessary remount the sensor, document the coolant change, and add material-lot hardness as a feature or stratification variable. Build a labeled dataset covering the old and new conditions with confirmed breaks and hard negatives. Hold out data by time and by machine, not randomly, so the validation reflects future operation.
Retrain only if the data supports it. Promote the model through shadow mode on one machine, with targets such as recall at least 0.90, false alarms below one per shift, and no missed breaks in the trial. Freeze the threshold and document the change. If those targets are not met, keep the existing model in advisory mode and rely on scheduled tool changes until the evidence improves.