Routing Repair Tickets at a Regional Water Utility
Northvale Water, a regional utility, handles about 52,000 repair tickets a year across six depots. Dispatchers currently route each ticket with a keyword list and customer-type rules. An internal audit found that 21% of tickets reach the wrong crew and sit idle for an average of 2.3 days before being rerouted. The operations manager wants a supervised model that predicts the correct crew from the free-text description, asset ID, location, time of day and customer segment. Three years of history are available. Labels are the crew that finally closed each ticket, but 14% of tickets were manually rerouted at least once, and emergency tickets are sometimes pre-assigned by a dispatcher before closure. The team has eight weeks, one data scientist, no GPU and a works council that demands explainable decisions. A vendor has offered an untested deep-learning model. The manager asks whether machine learning is justified and what evidence would be needed before a pilot.
1. Is supervised machine learning the right approach for this routing decision, and what exactly should the model predict? Justify your answer against the data and constraints.
Read the worked answer
Supervised learning is defensible only if the label represents the correct crew, not merely the crew that happened to close the ticket. As stated, the label is contaminated: 14% of tickets were rerouted, and emergency pre-assignment means the final crew may reflect dispatcher intervention rather than an objective routing rule. A model trained on final crew would learn to reproduce current human behaviour, including its errors.
I would define the target as the crew that should own the ticket, derived from an adjudicated sample of, say, 1,500 tickets reviewed by dispatchers and crew leads. If adjudication is too slow, I would first model reroute risk as a separate supervised problem and use it to prioritise human review, while keeping rules for emergency tickets. The problem is multiclass classification with text and tabular inputs; accuracy is not the right metric because crew volumes are uneven. The eight-week, one-data-scientist constraint also argues for a simple baseline and a limited pilot, not an untested deep model. If adjudicated labels cannot be produced, the honest conclusion is that ML may not be justified yet.
2. Identify the main data-quality and leakage risks in the feature set and labels, and explain how each could mislead the pilot.
Read the worked answer
The final-crew label includes human corrections and emergency overrides, so the model can learn noise as if it were truth. Dispatcher pre-assignment is especially risky: if it is recorded before closure and used as a feature, it leaks the outcome. High-cardinality asset IDs and depot locations may let the model memorise particular assets or local practices rather than learn transferable routing logic. Free-text fields need timestamp checks because resolution notes may be added after dispatch, effectively revealing the answer. Customer segment may proxy emergency status and raise fairness questions. Class volumes vary by crew, so overall accuracy would be dominated by the busiest crews. Seasonal and process drift mean a random split would overstate performance. Each of these can make the pilot look better offline while degrading in live routing.
3. Design an evaluation and pilot plan that would convince the works council and operations manager. Include metrics, split strategy and a stop criterion.
Read the worked answer
Use a time-based split: train on the first 24 months, validate on the next 6 and test on the final 6, grouping by asset or location so the same asset does not appear in training and test. Compare against the existing rule baseline on the same period. The primary metric should be a cost-weighted reroute rate, with macro-F1 and per-crew recall as secondary; also track emergency response time so routing gains do not harm safety. Run a four-week shadow-mode pilot in which dispatchers see suggestions but keep authority, and log every override. For explainability, show the top three text and tabular drivers per ticket and let the works council audit samples. Stop the pilot if there is no credible reduction in reroutes, any increase in emergency delay, or performance below baseline in two depots. Only then consider assisted routing.