MAS-II common mistakes
MAS-II · Understand the traps and how to avoid them.
When a GLM output shows a dispersion parameter far from 1, the candidate reports standard errors and confidence intervals straight from the model summary instead of rescaling them by the estimated dispersion (or refitting with a quasi-likelihood / negative binomial / Tweedie variance).
Software prints a coefficient table regardless of the variance assumption, and candidates treat the printed SEs as final. They conflate the mean-model fit with the variance-model fit, forgetting that overdispersion inflates the true variance of the coefficient estimates by roughly the dispersion factor.
What to do instead
Before interpreting any GLM output, check the estimated dispersion (Pearson chi-square over residual df, or the reported scale). If it departs materially from 1 for a Poisson or binomial model, refit with an appropriate variance structure and recompute Wald and likelihood-ratio tests. Practice computing the scale-corrected SE by hand once so the relationship is concrete.
The candidate selects a distribution or link by eyeballing the histogram of the raw response rather than by evaluating deviance residuals, the mean-variance relationship, and the range constraint of the response (e.g., fitting a Gaussian model to a strictly positive skewed severity, or a log link without checking the implied variance function).
Introductory courses emphasize distributional shape of Y, but GLM theory specifies the variance as a function of the mean. Candidates carry over the 'transform Y until normal' habit from linear models and never connect the link choice to the variance assumption.
What to do instead
Adopt a fixed diagnostic sequence: (1) identify the support and mean-variance relationship of the response, (2) choose the family from that, (3) choose the link from the scale on which effects are plausibly additive, (4) validate with deviance and randomized quantile residuals plus a binned residual plot against fitted values. Write the sequence down and use it on every problem.
In credibility and mixed-model problems, the candidate plugs the sample mean and sample size into the Bühlmann formula without first estimating the between-group variance component and the within-group variance, or uses the raw number of observations when exposure or effective sample size is the correct weight.
The credibility factor looks like a simple plug-in formula, so candidates skip the variance-component estimation step that the exam actually tests. They also miss that the structural parameters must be estimated from the data, not assumed.
What to do instead
Always write the three structural quantities explicitly (collective mean, expected process variance, variance of hypothetical means) and show how each is estimated from the portfolio. Distinguish number of observations from exposure and from effective degrees of freedom. Redo at least one full nonparametric empirical Bayes example end to end.
The candidate treats a fitted model's point predictions as final and never produces prediction intervals or quantiles that reflect both parameter uncertainty and process variance, or reports a mean prediction when the question asks for a VaR-type or CTE-type risk measure.
Simulation and prediction tooling returns a point estimate by default, and candidates do not separate estimation error from the stochastic error of a future observation. They also gloss over the difference between a confidence interval for the mean response and a prediction interval for a new draw.
What to do instead
For every prediction task, state which of the three sources of uncertainty is required: parameter, process, or both. For risk measures, derive the full predictive distribution (simulate from the fitted model, or use the closed form for the chosen family) and read off the quantile or tail expectation. Practice converting a predictive CDF into a CTE by integration or averaging.
When comparing nested models, the candidate compares AIC or BIC values, or compares deviances across models fitted to different subsets of data, different offsets, or different response transformations, and declares one model superior on an invalid comparison.
Likelihood-based criteria are only comparable on identical data and identical response scale. Candidates compute AIC routinely without checking that the offset, weights, and rows are held fixed, and they use AIC where a formal likelihood-ratio or score test is the intended tool.
What to do instead
Before any comparison, verify that the response, offset, weights, and observation set are identical. Use likelihood-ratio tests for nested models, and reserve AIC/BIC for non-nested or penalty-based selection. When a transformation changes the response scale, apply the Jacobian adjustment or avoid the comparison. Build a one-page checklist of comparability conditions.
The candidate fits a model with a categorical predictor and interprets the coefficient on one dummy level as the effect of that level, ignoring that it is a contrast against the reference level, and reports a main effect in a model containing an interaction as if it were a marginal effect averaged over the other variable.
Coefficient tables present main effects and interactions in one block, and the reference-level coding is invisible unless the candidate checks the contrasts. The habit of reading each row as an isolated effect survives from simple regression.
What to do instead
Always identify the reference level and write out the linear predictor for two contrasting cases before interpreting any coefficient. In the presence of interactions, compute predicted values or marginal effects at representative points rather than reading main-effect rows. Practice by reconstructing fitted values by hand from a coefficient table.
In survival and loss-timing problems, the candidate treats censored observations as if the event occurred at the censoring time, or drops them entirely, and computes a naive average time-to-event from the observed durations.
Censoring is a data-structure issue rather than a formula issue, and candidates default to complete-case arithmetic. They also confuse the hazard, the survival function, and the cumulative incidence, and apply the Kaplan-Meier product limit without accounting for heavy ties or for left truncation.
What to do instead
State the censoring mechanism explicitly at the start of every survival problem and confirm which observations are at risk in each interval. Derive the Kaplan-Meier estimator by constructing the risk set table by hand, then handle ties and truncation. Convert between hazard, survival, and cumulative hazard using the product-integral relationship until it is automatic.
The candidate runs a simulation study or bootstrap but draws the resampling units at the wrong level (resampling individual claims when the model has a policy-level random effect), or uses too few replicates, and then reports Monte Carlo noise as a substantive finding.
Resampling code is easy to write and hard to verify, and the clustering structure of the data is not visible in a flat data frame. Candidates also underestimate how many replicates are needed for stable tail estimates and standard errors.
What to do instead
Before running, write down the exchangeability unit implied by the model and resample at exactly that level (cluster bootstrap for hierarchical data, parametric bootstrap when the model is the assumed truth). Choose the replicate count from a target Monte Carlo standard error, and report that standard error alongside every simulated quantity. Validate the bootstrap on a case with a known analytic answer.
When a model fails to converge or produces implausible coefficients, the candidate reports the output anyway, or silently changes the optimizer settings until something converges, without diagnosing separation, collinearity, or a mis-specified link.
Convergence warnings are treated as cosmetic, and candidates lack a diagnostic vocabulary for non-convergence. Complete or quasi-complete separation in a logistic model, and near-collinear predictors, produce large coefficients that look like strong effects.
What to do instead
Treat any convergence warning as a stop signal. Check for separation (cross-tabulate the binary response against each categorical predictor), compute condition numbers or variance inflation for the design matrix, and inspect the fitted probability range. If separation is present, use a penalized or bias-reduced estimator and say so. Document the diagnostic in the answer.
The candidate answers a pricing or reserving question with a model-selection discussion when the question asks for a specific numerical quantity, or conversely gives a bare number with no statement of the assumptions, the estimator, and the uncertainty, so partial credit on the reasoning is lost.
Exam time pressure pushes candidates toward whichever response is faster to write, and the two failure directions attract different candidates: the analytically strong one under-shows work, the verbally strong one avoids the computation. Neither matches the grading rubric, which rewards both the number and its justification.
What to do instead
Use a fixed answer template: state assumptions and the estimator, show the computation with intermediate values, give the final number with units, then add one or two sentences on sensitivity or limitations. Rehearse the template on timed practice items so it fits the time budget without truncating the reasoning.