Methodology
Domain 2: Methodology
This domain tests selecting an analytic method that fits the framed problem, data realities, and decision constraints. CAP questions probe the descriptive-diagnostic-predictive-prescriptive continuum, method assumptions, validation design, and trade-offs such as interpretability versus accuracy.
Core rules and concepts
- Classify the question: descriptive (what happened), diagnostic (why), predictive (what will happen), prescriptive (what should be done). The class narrows the method family.
- Match method families to the question: regression and classification for prediction, clustering for segmentation, time series for temporal forecasts, optimization and simulation for prescriptive decisions, design of experiments for causal questions.
- State assumptions and check them: independence, linearity, stationarity, distributional assumptions, and data leakage risks. A method that violates assumptions needs remediation or a different approach.
- Choose validation that mirrors deployment: holdout, cross-validation, time-based splits for forecasting, and uplift or experimental designs where causal impact matters.
- Compare against a baseline: current rule, naive forecast, or simple model. Complexity must earn its keep in business value and maintainability.
- Use process frameworks such as CRISP-DM to organize phases, but let the decision context drive iteration.
Worked example
A call center wants to reduce average wait time. Descriptive analysis shows wait by hour and channel; diagnostic analysis identifies staffing gaps at lunch and after outages; predictive forecasting estimates call volume by 15-minute interval; prescriptive scheduling uses an optimization model with service-level constraints. Because the decision is staff scheduling under uncertainty, forecast plus optimization is appropriate. A simulation can test robustness to absenteeism. The team validates the forecast with time-based backtesting and compares schedule performance against the current rule using historical replay. A deep learning model might improve volume accuracy slightly, but if it cannot be explained to workforce planners or updated reliably, it fails the deployment constraint.
Traps that cost marks
- Selecting a sophisticated method before confirming the decision, data, and operating constraints.
- Ignoring assumptions or using random cross-validation on time series.
- Reporting accuracy without a baseline or business-relevant metric.
- Confusing correlation with causation when the decision requires a causal effect.
- Overfitting the method to available data rather than the deployment environment.