IFoA SA10 common mistakes
IFoA SA10 · Understand the traps and how to avoid them.
Writing a full data-science essay on model choice or code mechanics while giving only a sentence or two to the actuarial judgement, commercial implications, and recommendation the marker is actually awarding marks for.
Candidates come from a technical background and enjoy demonstrating method knowledge; under time pressure the technical section feels productive, so the professional/commercial discussion is left until the end and is truncated.
What to do instead
Before writing, allocate marks to minutes and reserve at least a third of the time for the judgement section. Structure each long answer as: business problem, method and why, results, limitations, recommendation to the decision-maker. Practise past-style questions with a hard timer and mark your own script against the command verbs to see where the judgement marks were lost.
Presenting a model result without stating the assumption, the validation performed, or the uncertainty around it - for example quoting a predicted claim frequency to three decimal places with no confidence interval, no hold-out or cross-validation evidence, and no discussion of data quality.
Course material and software output encourage a single point estimate; candidates confuse a fitted value with a validated finding and assume the marker will infer the caveats.
What to do instead
Adopt a standing checklist for any model discussion: data source and known quality issues, assumptions, validation method and metric, uncertainty (interval or sensitivity range), and how the result would change the decision. Explicitly write the caveat sentences - unstated reasoning earns no marks.
Confusing supervised and unsupervised techniques, or misapplying a method to the wrong data type - e.g. proposing a clustering algorithm for a labelled fraud-detection problem, using accuracy as the metric on heavily imbalanced claim or fraud data, or treating time-ordered experience as an i.i.d. random sample.
The syllabus covers a wide method landscape and candidates learn definitions in isolation without linking each method to the data structure and objective it suits; exam pressure surfaces the most recently revised method regardless of fit.
What to do instead
Build a one-page method-selection table: problem type (prediction, classification, segmentation, dimension reduction), target availability, data volume, interpretability need, and the appropriate validation metric. For classification, default to precision/recall, AUC or a cost-weighted measure and say why accuracy is misleading when classes are rare.
Ignoring the regulatory and ethical framework in the UK context - failing to mention the PRA/FCA expectations on model risk management, the Consumer Duty fair-value outcome, explainability for affected customers, or the treatment of protected characteristics and proxy variables under equality law.
Candidates treat ethics as a generic paragraph about 'being fair' and do not connect it to named UK obligations; they also underestimate how heavily SA10 rewards linking data-science choices to specific regulatory duties.
What to do instead
Prepare short, memorised anchors: model risk governance (documentation, independent validation, ongoing monitoring), Consumer Duty outcomes, GDPR lawful basis and data minimisation, explainability and contestability for automated decisions, and bias testing including proxy discrimination. Apply the relevant anchor by name in each scenario answer rather than writing a general fairness paragraph.
Describing a machine-learning technique in textbook terms without translating it into actuarial workflow - i.e. never saying where in the pricing, reserving, capital or underwriting process the output would be used, who consumes it, or how it would be implemented and monitored in production.
The domain 'data science applications in actuarial work' is examined through scenarios, but candidates revise methods as theory and run out of time to discuss integration, so answers stay abstract.
What to do instead
For every technique you revise, write two sentences: one on the actuarial task it improves and one on the operational consequence (data feeds, run frequency, model ownership, monitoring triggers, downstream reporting). In the exam, explicitly name the process step and the decision it informs.
Failing to distinguish correlation from causation or to identify confounding and leakage - for example concluding that a risk factor 'causes' higher claims from observational portfolio data, or building features from information not available at the point of decision (post-event data leakage).
Real insurance datasets are observational and messy; candidates are trained to fit models and report associations, and leakage is invisible unless you deliberately audit feature timing.
What to do instead
For any claimed relationship, state the alternative explanations (confounding, selection, reverse causality) and what evidence would distinguish them (randomised or natural experiment, temporal validation, hold-out by time). Audit every feature against the decision timestamp and say so in the answer.
Treating data preparation as a trivial preliminary - omitting discussion of missing values, outliers, class imbalance, feature scaling, train/test splitting, and how these choices affect the model and the conclusions drawn from it.
Candidates focus revision on algorithms and assume data cleaning is common sense; in scenario questions the dataset problems are the substance being tested.
What to do instead
Write a standard data-diagnostics paragraph for scenarios: completeness, consistency, granularity, representativeness, missingness mechanism (and whether imputation is defensible), outliers, and the validation split design. State the consequence of each issue for the model and for the business decision.
Answering a 'critically evaluate' or 'discuss' command with a one-sided list of advantages, omitting disadvantages, alternatives, and a reasoned conclusion; or answering a 'calculate and interpret' question with the number only.
Under time pressure candidates dump knowledge in bullet lists and stop when they run out of points, rather than answering the verb; the interpretative sentence is the part that carries the higher-order marks.
What to do instead
Underline the command verb before planning. For evaluate/discuss, force a two-column plan (benefits versus drawbacks) plus a conclusion sentence that commits to a position. For calculations, always follow the number with a sentence on what it means, its sensitivity, and its implication for the decision.
Producing vague, unquantified recommendations - 'the model should be monitored regularly' or 'more data would help' - with no metric, threshold, frequency, owner, or cost-benefit comparison against the status quo.
Candidates know monitoring and governance are expected but have not rehearsed concrete content, so they write the heading without the substance and the marker cannot award application marks.
What to do instead
Memorise concrete monitoring content: population stability index and characteristic drift thresholds, discrimination and calibration tracking, back-testing frequency, trigger-based revalidation, named model owner and independent review. Quantify where possible and compare the proposed approach with the existing process on cost, accuracy and risk.
Misallocating time across the paper - over-running on an early question to perfect a technical derivation, then leaving a whole later question or its final judgement section unanswered, or writing long preambles that repeat the scenario.
Marks are broadly comparable across questions, but a difficult early question feels like the place to recover marks; candidates also draft extensive introductions instead of answering directly.
What to do instead
Do a two-minute scan and attempt questions in order of confidence, keeping strictly to marks-proportional time and leaving space to return. Start each answer with the direct point rather than restating the question, and practise full papers under exam conditions so the pacing is rehearsed, not improvised.