Scoping a Regulated Demand Forecasting Platform
A regional energy utility wants day-ahead demand forecasts for 42 substations. Its current spreadsheet process produces an 11% MAPE, and the VP of Operations has promised regulators a 7% MAPE within nine months. Constraints: the regulator requires seven years of model lineage and explainable drivers for rate cases; customer data cannot leave the country; the platform budget is $600k; the team is two data scientists and one platform engineer. A SaaS forecasting vendor can deliver in six weeks but hosts outside the country. An on-prem build on existing Intel Xeon servers with Kubernetes, MLflow, and Kubeflow is feasible but slower. Stakeholders also disagree about real-time scoring, although forecasts are only consumed once daily at 16:00. About 30% of substations have less than two years of history, and three have known meter replacement gaps.
1. Recommend an MVP scope and target architecture. Explain which constraints drive your choices and what you would explicitly defer.
Read the worked answer
Start from the hard constraints, not the tools. Data residency and seven-year lineage make an in-country deployment non-negotiable, so the SaaS option is out unless the vendor offers an in-country VPC. Real-time scoring is not a requirement: forecasts are consumed once daily at 16:00, so a batch pipeline is sufficient. A defensible MVP is nightly ingestion, a global gradient-boosted model with substation-level features and a hierarchical fallback for the 30% short-history sites, MLflow for lineage, and a feature table with as-of timestamps. Use Intel Extension for scikit-learn on the existing Xeon fleet to accelerate training without new hardware. Defer streaming, deep learning, and per-substation bespoke models. Choose metrics that match the regulator: MAPE and pinball loss for intervals, plus explainability via SHAP for rate-case drivers. The key judgement is that the 7% target is unlikely in nine months for short-history substations; set a realistic tiered target and make the gap explicit to the VP.
2. The CIO asks whether to buy the SaaS forecasting tool anyway. Define the decision criteria and a go/no-go gate.
Read the worked answer
Treat this as an architecture decision with one non-negotiable constraint and several trade-offs. First test the vendor against data residency and lineage: can it run in-country, export seven years of model artefacts, and provide explanations the regulator accepts? If no, the decision is closed. If yes, compare total cost over three years, time-to-value, lock-in, and integration with the existing identity and monitoring stack. A weighted scorecard with residency as a gate is more defensible than a feature checklist. Run a two-week bake-off on the same 42 substations: vendor versus an on-prem baseline using the same features. Set a go/no-go gate at week six: the on-prem path must reach at most 9.5% MAPE on a held-out quarter, or the team pivots to the vendor. Document the gate, the metric, and the decision owner before the bake-off so the choice is not driven by the loudest stakeholder.