Data Quality Issues in Predictive Maintenance
A manufacturing plant uses sensors on critical machinery to predict failures. The data pipeline collects temperature, vibration, and pressure readings every minute. Recently, the predictive model's accuracy dropped from 92% to 75%. An investigation found: (1) 15% of sensor readings are missing due to network outages; (2) some sensors are miscalibrated, reporting values outside normal range; (3) maintenance logs are inconsistent, with technicians using free-text notes; (4) the model was trained on data from only one machine type, but now two new machine types are in use. The plant operates 24/7, and unplanned downtime costs $10,000 per hour. The data science team wants to quickly restore accuracy.
1. Diagnose the data quality issues and their impact on model performance.
Read the worked answer
The accuracy drop is likely due to multiple data quality issues: 1. Missing data (15%): Network outages cause gaps. If missingness is not random (e.g., during peak load), it can bias the model. Imputation methods may introduce errors. 2. Miscalibrated sensors: Out-of-range values can be outliers that distort model training and inference. They may also indicate actual failures, but if miscalibrated, they are noise. 3. Inconsistent maintenance logs: Free-text notes are unstructured and may not be reliably parsed. This affects labels for supervised learning; if labels are noisy, model learns incorrect patterns. 4. Training-serving skew: Model trained on one machine type may not generalize to new types due to different vibration patterns, etc. This is a distribution shift. The combination reduces model's ability to detect true failures, increasing false positives/negatives. The cost of downtime means false negatives (missed failures) are costly.
2. Propose a remediation plan to address these issues and restore model accuracy, including data governance steps.
Read the worked answer
Remediation plan: 1. Data quality monitoring: Implement real-time checks for missing values and out-of-range readings. Set alerts for anomalies. 2. Missing data handling: Use appropriate imputation (e.g., forward-fill for short gaps, model-based imputation for longer gaps) and flag imputed values. Consider collecting data on outage causes to assess missingness mechanism. 3. Sensor calibration: Schedule regular calibration; use statistical process control to detect drift. Replace faulty sensors. 4. Standardize maintenance logs: Introduce structured forms with dropdowns and standardized codes for failure types. Use NLP to extract information from historical free-text notes. 5. Model retraining: Collect data from new machine types and retrain the model, possibly using transfer learning or separate models per type. 6. Data governance: Establish data ownership, quality metrics, and regular audits. Document data lineage. 7. Pilot testing: Validate the updated model on a holdout set before full deployment. 8. Human oversight: Allow maintenance staff to override predictions. 9. Cost-benefit: Given downtime cost, invest in redundant network and sensors.