CSPA common mistakes
CSPA · Understand the traps and how to avoid them.
Combining incompatible insurance ratios
Frequency, severity, exposure, expense, and coverage bases can differ even when each individual number is valid.
What to do instead
Write the numerator, denominator, dates, and population for every component before applying an identity.
Confusing paid, reported, and ultimate amounts
A payment total, reported incurred figure, and estimated ultimate loss describe different stages or definitions.
What to do instead
Use the problem definitions and reconcile the stated reserve and development quantities before calculating.
Learning preprocessing from the held-out data
Even apparently harmless imputation or selection can let evaluation information shape the fitted pipeline.
What to do instead
Fit every learned transformation within its training split and apply the fitted mapping consistently.
Using future or late-entered features
An event date before renewal does not establish that the information was available when the score was produced.
What to do instead
Document feature availability timestamps and reconstruct the actual scoring-time view.
Treating an odds ratio as a probability ratio
Logistic coefficients multiply odds, while probabilities require a separate conversion that depends on the baseline.
What to do instead
Convert baseline probability to odds, multiply, and convert back before reporting the result.
Ignoring interactions or offsets
A main coefficient is conditional when interactions exist, and an offset has a specified role rather than a freely fitted effect.
What to do instead
Write the full linear predictor and identify fixed contributions before interpreting a mean or rate change.
Equating training fit with generalization
Deep trees and long boosting runs can adapt strongly to the training data without improving future predictions.
What to do instead
Tune complexity using a suitable validation design and compare with simpler defensible baselines.
Calling an explanation causal evidence
Importance, local contributions, and partial-dependence summaries describe a fitted model rather than proving an intervention effect.
What to do instead
Describe the actual explanation method, correlation limitations, reference data, and causal assumptions separately.
Choosing a split that does not match deployment
Random rows can share people, events, or time information that would not be available in the intended future setting.
What to do instead
Specify the target deployment population and preserve relevant groups, time ordering, and outcome maturity.
Treating AUC as calibration or business value
A ranking improvement does not guarantee better probabilities, fewer costly errors, or a higher-value constrained queue.
What to do instead
Report the relevant discrimination, calibration, decision, subgroup, and operational measures together.
Presenting a fraud score as proof
A predictive score is not an investigation finding or evidence establishing intent for a specific person.
What to do instead
Use the score to support authorized review with appropriate evidence, safeguards, and communication.
Claiming reproducibility from a seed alone
Data, feature code, package behavior, and split logic can change while a random seed remains the same.
What to do instead
Record controlled versions and preserve an executable, documented pipeline rather than only the final metric.