CDSP (DSP-110) cheat sheet

A business problem becomes an analytics problem only when it specifies a decision, an actor, a metric, and a time horizon; missing any element means the scope is not yet defined.

SMART framing check: a goal must name a Specific metric, be Measurable from available data, have an Agreed owner, be Relevant to a decision, and carry a Time bound.

Root-cause discipline: separate symptom (metric movement) from cause (mechanism) from decision (action); a problem statement that jumps to a solution is a solution-first bias.

Decision-vs-exploration rule: if the output feeds a repeatable choice, treat it as supervised/predictive; if it maps unknown structure, treat it as unsupervised/descriptive; if it allocates a constrained action, treat it as optimization.

KPI ladder: define north-star metric, then driver metrics, then guardrail metrics that must not degrade; every model objective maps to one rung of this ladder.

Cost-of-error framing: quantify false positive cost and false negative cost separately; the ratio sets the classification threshold and the acceptable error balance.

Baseline rule: state the current process performance (status quo) before any modeling; a model must beat this baseline to justify its cost.

Assumption log: record each assumption with its owner and the test that would falsify it; unresolved assumptions become explicit risks in the final recommendation.

Scope contract: list in-scope entities, time window, granularity, and exclusions in writing; ambiguity in any of these is the most common cause of rework.

Missing-data mechanisms: MCAR (missingness independent of all values), MAR (dependent on observed values, so imputation is valid), MNAR (dependent on the unobserved value itself, so imputation biases results).

Imputation choices: mean/median for simple numeric, mode for categorical, model-based (regression, k-NN) for MAR with strong predictors, and a missing-indicator flag when missingness itself is informative.

Outlier detection rules: IQR fence at Q1 - 1.5IQR and Q3 + 1.5IQR; z-score threshold |z| > 3; Mahalanobis distance for multivariate outliers; treat as error, natural variation, or signal before deciding.

Leakage controls: fit scalers, encoders, and imputers inside cross-validation folds only; any statistic computed on the full dataset before splitting leaks target information.

Class imbalance: resampling (SMOTE, undersampling) changes the prior, so recalibrate probabilities afterward; class weights and threshold tuning usually preserve calibration better.

Duplicate handling: exact duplicates are removed; near-duplicates require a similarity key (e.g., normalized name + date) and a documented merge rule with survivorship logic.

Data quality dimensions to check: completeness, uniqueness, validity (range/format), consistency (cross-field), timeliness, and accuracy against a trusted source.

Normalization vs standardization: min-max scaling maps to [0,1] and is sensitive to outliers; z-score standardization centers at 0 with unit variance and assumes roughly symmetric spread.

Train/validation/test split rule: test set is touched once at the end; with time-ordered data use chronological splits, never random shuffles, to avoid temporal leakage.

Bias-variance tradeoff: high bias shows as both train and validation error being high; high variance shows as low train error with a large validation gap; remedies are more features/complexity vs regularization/more data respectively.

Metric selection rule: accuracy is misleading under imbalance; use precision/recall or F1 when the positive class matters, ROC-AUC for ranking across thresholds, PR-AUC when positives are rare, and log loss when calibrated probabilities are needed.

Regression metrics: RMSE penalizes large errors and is in target units; MAE is robust to outliers; R-squared is relative to the mean baseline and can be negative on held-out data.

Cross-validation design: k-fold for i.i.d. data, stratified k-fold for classification, grouped k-fold when entities repeat, and time-series split for temporal data.

Regularization: L1 (lasso) drives coefficients to zero and performs selection; L2 (ridge) shrinks smoothly; elastic net blends both and is preferred with correlated predictors.

Interpretability tradeoff: linear and tree-based models give coefficients or split logic; ensembles and neural nets need post-hoc explanation (permutation importance, SHAP), which explains the model, not causality.

Assumption checks: linear regression needs linearity, independence, homoscedasticity, and normality of residuals; violations point to transformation, robust standard errors, or a different model family.

Model comparison rule: compare on the same validation scheme and same metric; a difference smaller than fold-to-fold variance is not evidence of superiority.

Overfitting signals: validation metric degrades while training metric improves, learning curve plateaus early at a low level, or performance collapses under slight data perturbation.

Confidence interval reading: a 95% interval means the procedure captures the true parameter in 95% of repeated samples, not that the parameter has a 95% probability of lying in this interval.

p-value definition: probability of data at least as extreme as observed, given the null hypothesis is true; it is not the probability the null is true nor the probability of replication.

Statistical vs practical significance: with large samples tiny effects become significant; report effect size and its business-relevant threshold alongside the p-value.

Correlation limits: correlation measures linear association only, is symmetric, and is confounded by a third variable; causal claims require design (experiment, natural experiment) or explicit assumptions.

Simpson's paradox: an association can reverse when data are aggregated across groups; always check the effect within key segments before reporting an overall direction.

Multiple comparisons: testing many hypotheses inflates false positives; apply Bonferroni (alpha/m) or Benjamini-Hochberg false discovery rate control.

Experiment interpretation: distinguish statistical power failures (inconclusive) from genuine nulls; check for novelty effects, sample ratio mismatch, and guardrail metric movement.

Extrapolation warning: a model's validity is bounded by the predictor range and population seen in training; predictions outside that support region are unvalidated.

Segment analysis discipline: pre-specify segments; post-hoc slicing multiplies comparisons and produces spurious findings that rarely replicate.

Answer-first structure: lead with the decision-relevant conclusion, then evidence, then method; executives stop reading after the first paragraph if the point is buried.

Audience mapping: executives need decision, impact, and risk; analysts need method and assumptions; operators need the action and its trigger conditions - one deck rarely serves all three.

Quantify impact in business units (revenue, cost, hours, risk exposure) with an explicit uncertainty range, not just model metrics like AUC or RMSE.

Chart selection rule: bar for category comparison, line for trend over time, scatter for two-variable relationship, histogram for distribution; pie charts only for parts of a whole with few slices.

Uncertainty communication: pair every point estimate with an interval and state what would change the recommendation; avoid single-number forecasts without ranges.

Recommendation format: state the action, the expected effect with confidence, the cost or risk, the owner, and the trigger for revisiting the decision.

Avoiding misleading visuals: truncated y-axes exaggerate differences, dual axes imply relationships, and cherry-picked time windows distort trends; label axes and state the window.

Pre-registration of success criteria: agree on the metric and threshold before analysis so results are judged against a fixed standard, not a post-hoc one.

Limitations disclosure: name the key assumptions, data gaps, and the population the result does not cover; unstated limitations become the stakeholder's surprise later.