IFoA SA10 cheat sheet

Data science pipeline: problem framing, data acquisition, cleaning and validation, exploratory analysis, modelling, validation, deployment, and monitoring; each stage has actuarial controls for reproducibility and auditability.

Data types matter: structured (tabular, relational), semi-structured (JSON, XML, logs), and unstructured (text, images, audio); actuarial models typically assume structured numeric or categorical inputs, so unstructured sources require feature extraction first.

Bias-variance decomposition: expected squared error = bias squared + variance + irreducible noise; increasing model complexity lowers bias and raises variance, so the tuning objective is total error, not training fit.

Train/validation/test split logic: fit parameters on train, tune hyperparameters on validation, report performance once on test; leakage occurs when information from validation or test rows enters training (e.g. scaling before splitting).

Cross-validation: k-fold gives k estimates of out-of-sample performance; stratified k-fold preserves class balance for rare events, and time-series data needs rolling-origin or blocked splits to avoid look-ahead bias.

Class imbalance handling: resampling (over/under), class weights, and threshold tuning change the decision boundary but not the underlying data-generating process; report precision, recall, and calibration, not accuracy alone.

Missing data mechanisms: MCAR (missing completely at random), MAR (missing at random given observed variables), MNAR (missing not at random); complete-case analysis is unbiased only under MCAR, while multiple imputation is valid under MAR.

Reproducibility requires recorded data versions, code, seeds, package versions, and environment; without these, a model cannot be independently re-run, which conflicts with actuarial documentation and peer-review expectations.

Pricing and underwriting: GLMs remain the regulatory baseline in many markets; gradient boosting and neural networks are used for risk segmentation but often need explainability surrogates or constrained monotonic relationships.

Claims reserving: machine learning supplements chain-ladder and Bornhuetter-Ferguson by modelling claim-level development; individual claim models can capture case-estimate behaviour and settlement lags that aggregate triangles miss.

Fraud and anomaly detection is a rare-event problem: precision-recall curves and lift charts are more informative than ROC curves, and false-positive cost (customer friction) must be weighed against claim savings.

Mortality and longevity: Lee-Carter and its extensions model log mortality with an age profile and a time index; regularised and machine learning variants forecast the time index or allow richer age-period interactions.

Customer retention and lifetime value: churn models feed survival or hazard models; expected lifetime value = expected duration x expected margin, so both components need separate validation.

Capital and risk modelling: machine learning metamodels approximate slow simulation engines (e.g. nested stochastic), but the surrogate must be validated on out-of-sample scenarios and checked for tail behaviour.

Text and document analytics: natural language processing supports claims triage, complaint classification, and policy wording comparison; performance is judged by precision and recall per class, not aggregate accuracy.

Deployment and monitoring: population stability index and drift tests on input features, plus periodic recalibration, detect degradation; model risk management requires documented ownership, change control, and retirement criteria.

Generalised linear models: link function g(mu) = linear predictor, variance proportional to a power of the mean; Poisson (counts), gamma (severity), and Tweedie (aggregate loss with mass at zero) are standard actuarial families.

Regularisation: ridge adds an L2 penalty and shrinks coefficients, lasso adds an L1 penalty and performs selection, elastic net combines both; penalty strength is chosen by cross-validation, and features should be scaled first.

Decision trees split on impurity reduction (Gini or entropy for classification, variance for regression); single trees are unstable, so ensembles (bagging, random forests, boosting) trade interpretability for accuracy.

Gradient boosting builds additive trees sequentially, each fitting the negative gradient of the loss; key hyperparameters are tree depth, learning rate, number of trees, and row/column subsampling.

Neural networks: layered linear transforms with non-linear activations; training minimises a loss by gradient descent, with dropout and early stopping as regularisers, and they require large data and careful tuning.

Model evaluation: ROC-AUC measures ranking across thresholds, log loss measures calibration and sharpness, and Gini = 2 x AUC - 1; for regression use RMSE, MAE, and deviance residuals.

Calibration: a model is calibrated if predicted probabilities match observed frequencies; check with reliability diagrams, Hosmer-Lemeshow style tests, or by fitting a logistic recalibration on held-out data.

Explainability tools: variable importance, partial dependence plots, and SHAP values attribute predictions to features; SHAP is additive per prediction but correlated features can split credit misleadingly.

Clustering and dimension reduction: k-means minimises within-cluster sum of squares and needs k chosen in advance; PCA projects onto orthogonal components ordered by explained variance and is sensitive to feature scaling.

Actuarial standards require models to be documented, validated, and communicated with appropriate caveats; using a model outside its validated scope is a professional failure regardless of technical accuracy.

Fairness: protected characteristics (e.g. age, sex, disability, ethnicity) are restricted or prohibited rating factors in many jurisdictions; proxy variables can reintroduce discrimination even when the protected field is excluded.

Explainability versus accuracy trade-off: complex models may be justified only where outcomes can be explained to customers, regulators, and the business, or where a documented surrogate explanation is acceptable.

Data ethics: purpose limitation, data minimisation, lawful basis, retention limits, and subject access rights under data protection law; secondary use of data for a new model may need fresh consent or assessment.

Algorithmic accountability: named model owner, independent validation, audit trail of data and code, and a clear escalation route for adverse findings; model risk frameworks grade models by materiality.

Transparency to customers: where automated decisions materially affect individuals, disclosure and, in some regimes, a right to human review or explanation apply; opacity is not a defence.

Uncertainty communication: present ranges and sensitivity, not single point estimates; overstating model precision to decision-makers is a professional integrity issue.

Emerging AI regulation: risk-tiered obligations (prohibited, high-risk, limited-risk uses) impose documentation, human oversight, and monitoring duties; actuarial work in pricing and claims may fall in higher tiers.

Data science pipeline: problem framing, data acquisition, cleaning and validation, exploratory analysis, modelling, validation, deployment, and monitoring; each stage has actuarial controls for reproducibility and auditability.

Data types matter: structured (tabular, relational), semi-structured (JSON, XML, logs), and unstructured (text, images, audio); actuarial models typically assume structured numeric or categorical inputs, so unstructured sources require feature extraction first.

Bias-variance decomposition: expected squared error = bias squared + variance + irreducible noise; increasing model complexity lowers bias and raises variance, so the tuning objective is total error, not training fit.

Train/validation/test split logic: fit parameters on train, tune hyperparameters on validation, report performance once on test; leakage occurs when information from validation or test rows enters training (e.g. scaling before splitting).

Cross-validation: k-fold gives k estimates of out-of-sample performance; stratified k-fold preserves class balance for rare events, and time-series data needs rolling-origin or blocked splits to avoid look-ahead bias.

Class imbalance handling: resampling (over/under), class weights, and threshold tuning change the decision boundary but not the underlying data-generating process; report precision, recall, and calibration, not accuracy alone.

Missing data mechanisms: MCAR (missing completely at random), MAR (missing at random given observed variables), MNAR (missing not at random); complete-case analysis is unbiased only under MCAR, while multiple imputation is valid under MAR.

Reproducibility requires recorded data versions, code, seeds, package versions, and environment; without these, a model cannot be independently re-run, which conflicts with actuarial documentation and peer-review expectations.

Pricing and underwriting: GLMs remain the regulatory baseline in many markets; gradient boosting and neural networks are used for risk segmentation but often need explainability surrogates or constrained monotonic relationships.

Claims reserving: machine learning supplements chain-ladder and Bornhuetter-Ferguson by modelling claim-level development; individual claim models can capture case-estimate behaviour and settlement lags that aggregate triangles miss.

Fraud and anomaly detection is a rare-event problem: precision-recall curves and lift charts are more informative than ROC curves, and false-positive cost (customer friction) must be weighed against claim savings.

Mortality and longevity: Lee-Carter and its extensions model log mortality with an age profile and a time index; regularised and machine learning variants forecast the time index or allow richer age-period interactions.

Customer retention and lifetime value: churn models feed survival or hazard models; expected lifetime value = expected duration x expected margin, so both components need separate validation.

Capital and risk modelling: machine learning metamodels approximate slow simulation engines (e.g. nested stochastic), but the surrogate must be validated on out-of-sample scenarios and checked for tail behaviour.

Text and document analytics: natural language processing supports claims triage, complaint classification, and policy wording comparison; performance is judged by precision and recall per class, not aggregate accuracy.

Deployment and monitoring: population stability index and drift tests on input features, plus periodic recalibration, detect degradation; model risk management requires documented ownership, change control, and retirement criteria.

Generalised linear models: link function g(mu) = linear predictor, variance proportional to a power of the mean; Poisson (counts), gamma (severity), and Tweedie (aggregate loss with mass at zero) are standard actuarial families.

Regularisation: ridge adds an L2 penalty and shrinks coefficients, lasso adds an L1 penalty and performs selection, elastic net combines both; penalty strength is chosen by cross-validation, and features should be scaled first.

Decision trees split on impurity reduction (Gini or entropy for classification, variance for regression); single trees are unstable, so ensembles (bagging, random forests, boosting) trade interpretability for accuracy.

Gradient boosting builds additive trees sequentially, each fitting the negative gradient of the loss; key hyperparameters are tree depth, learning rate, number of trees, and row/column subsampling.

Neural networks: layered linear transforms with non-linear activations; training minimises a loss by gradient descent, with dropout and early stopping as regularisers, and they require large data and careful tuning.

Model evaluation: ROC-AUC measures ranking across thresholds, log loss measures calibration and sharpness, and Gini = 2 x AUC - 1; for regression use RMSE, MAE, and deviance residuals.

Calibration: a model is calibrated if predicted probabilities match observed frequencies; check with reliability diagrams, Hosmer-Lemeshow style tests, or by fitting a logistic recalibration on held-out data.

Explainability tools: variable importance, partial dependence plots, and SHAP values attribute predictions to features; SHAP is additive per prediction but correlated features can split credit misleadingly.

Clustering and dimension reduction: k-means minimises within-cluster sum of squares and needs k chosen in advance; PCA projects onto orthogonal components ordered by explained variance and is sensitive to feature scaling.

Actuarial standards require models to be documented, validated, and communicated with appropriate caveats; using a model outside its validated scope is a professional failure regardless of technical accuracy.

Fairness: protected characteristics (e.g. age, sex, disability, ethnicity) are restricted or prohibited rating factors in many jurisdictions; proxy variables can reintroduce discrimination even when the protected field is excluded.

Explainability versus accuracy trade-off: complex models may be justified only where outcomes can be explained to customers, regulators, and the business, or where a documented surrogate explanation is acceptable.

Data ethics: purpose limitation, data minimisation, lawful basis, retention limits, and subject access rights under data protection law; secondary use of data for a new model may need fresh consent or assessment.

Algorithmic accountability: named model owner, independent validation, audit trail of data and code, and a clear escalation route for adverse findings; model risk frameworks grade models by materiality.

Transparency to customers: where automated decisions materially affect individuals, disclosure and, in some regimes, a right to human review or explanation apply; opacity is not a defence.

Uncertainty communication: present ranges and sensitivity, not single point estimates; overstating model precision to decision-makers is a professional integrity issue.

Emerging AI regulation: risk-tiered obligations (prohibited, high-risk, limited-risk uses) impose documentation, human oversight, and monitoring duties; actuarial work in pricing and claims may fall in higher tiers.

Data science pipeline: problem framing, data acquisition, cleaning and validation, exploratory analysis, modelling, validation, deployment, and monitoring; each stage has actuarial controls for reproducibility and auditability.

Data types matter: structured (tabular, relational), semi-structured (JSON, XML, logs), and unstructured (text, images, audio); actuarial models typically assume structured numeric or categorical inputs, so unstructured sources require feature extraction first.

Bias-variance decomposition: expected squared error = bias squared + variance + irreducible noise; increasing model complexity lowers bias and raises variance, so the tuning objective is total error, not training fit.

Train/validation/test split logic: fit parameters on train, tune hyperparameters on validation, report performance once on test; leakage occurs when information from validation or test rows enters training (e.g. scaling before splitting).

Cross-validation: k-fold gives k estimates of out-of-sample performance; stratified k-fold preserves class balance for rare events, and time-series data needs rolling-origin or blocked splits to avoid look-ahead bias.

Class imbalance handling: resampling (over/under), class weights, and threshold tuning change the decision boundary but not the underlying data-generating process; report precision, recall, and calibration, not accuracy alone.

Missing data mechanisms: MCAR (missing completely at random), MAR (missing at random given observed variables), MNAR (missing not at random); complete-case analysis is unbiased only under MCAR, while multiple imputation is valid under MAR.

Reproducibility requires recorded data versions, code, seeds, package versions, and environment; without these, a model cannot be independently re-run, which conflicts with actuarial documentation and peer-review expectations.

Pricing and underwriting: GLMs remain the regulatory baseline in many markets; gradient boosting and neural networks are used for risk segmentation but often need explainability surrogates or constrained monotonic relationships.

Claims reserving: machine learning supplements chain-ladder and Bornhuetter-Ferguson by modelling claim-level development; individual claim models can capture case-estimate behaviour and settlement lags that aggregate triangles miss.

Fraud and anomaly detection is a rare-event problem: precision-recall curves and lift charts are more informative than ROC curves, and false-positive cost (customer friction) must be weighed against claim savings.

Mortality and longevity: Lee-Carter and its extensions model log mortality with an age profile and a time index; regularised and machine learning variants forecast the time index or allow richer age-period interactions.

Customer retention and lifetime value: churn models feed survival or hazard models; expected lifetime value = expected duration x expected margin, so both components need separate validation.

Capital and risk modelling: machine learning metamodels approximate slow simulation engines (e.g. nested stochastic), but the surrogate must be validated on out-of-sample scenarios and checked for tail behaviour.

Text and document analytics: natural language processing supports claims triage, complaint classification, and policy wording comparison; performance is judged by precision and recall per class, not aggregate accuracy.

Deployment and monitoring: population stability index and drift tests on input features, plus periodic recalibration, detect degradation; model risk management requires documented ownership, change control, and retirement criteria.

Generalised linear models: link function g(mu) = linear predictor, variance proportional to a power of the mean; Poisson (counts), gamma (severity), and Tweedie (aggregate loss with mass at zero) are standard actuarial families.

Regularisation: ridge adds an L2 penalty and shrinks coefficients, lasso adds an L1 penalty and performs selection, elastic net combines both; penalty strength is chosen by cross-validation, and features should be scaled first.

Decision trees split on impurity reduction (Gini or entropy for classification, variance for regression); single trees are unstable, so ensembles (bagging, random forests, boosting) trade interpretability for accuracy.

Gradient boosting builds additive trees sequentially, each fitting the negative gradient of the loss; key hyperparameters are tree depth, learning rate, number of trees, and row/column subsampling.

Neural networks: layered linear transforms with non-linear activations; training minimises a loss by gradient descent, with dropout and early stopping as regularisers, and they require large data and careful tuning.

Model evaluation: ROC-AUC measures ranking across thresholds, log loss measures calibration and sharpness, and Gini = 2 x AUC - 1; for regression use RMSE, MAE, and deviance residuals.

Calibration: a model is calibrated if predicted probabilities match observed frequencies; check with reliability diagrams, Hosmer-Lemeshow style tests, or by fitting a logistic recalibration on held-out data.

Explainability tools: variable importance, partial dependence plots, and SHAP values attribute predictions to features; SHAP is additive per prediction but correlated features can split credit misleadingly.

Clustering and dimension reduction: k-means minimises within-cluster sum of squares and needs k chosen in advance; PCA projects onto orthogonal components ordered by explained variance and is sensitive to feature scaling.

Actuarial standards require models to be documented, validated, and communicated with appropriate caveats; using a model outside its validated scope is a professional failure regardless of technical accuracy.

Fairness: protected characteristics (e.g. age, sex, disability, ethnicity) are restricted or prohibited rating factors in many jurisdictions; proxy variables can reintroduce discrimination even when the protected field is excluded.

Explainability versus accuracy trade-off: complex models may be justified only where outcomes can be explained to customers, regulators, and the business, or where a documented surrogate explanation is acceptable.

Data ethics: purpose limitation, data minimisation, lawful basis, retention limits, and subject access rights under data protection law; secondary use of data for a new model may need fresh consent or assessment.

Algorithmic accountability: named model owner, independent validation, audit trail of data and code, and a clear escalation route for adverse findings; model risk frameworks grade models by materiality.

Transparency to customers: where automated decisions materially affect individuals, disclosure and, in some regimes, a right to human review or explanation apply; opacity is not a defence.

Uncertainty communication: present ranges and sensitivity, not single point estimates; overstating model precision to decision-makers is a professional integrity issue.

Emerging AI regulation: risk-tiered obligations (prohibited, high-risk, limited-risk uses) impose documentation, human oversight, and monitoring duties; actuarial work in pricing and claims may fall in higher tiers.

Data science pipeline: problem framing, data acquisition, cleaning and validation, exploratory analysis, modelling, validation, deployment, and monitoring; each stage has actuarial controls for reproducibility and auditability.

Data types matter: structured (tabular, relational), semi-structured (JSON, XML, logs), and unstructured (text, images, audio); actuarial models typically assume structured numeric or categorical inputs, so unstructured sources require feature extraction first.

Bias-variance decomposition: expected squared error = bias squared + variance + irreducible noise; increasing model complexity lowers bias and raises variance, so the tuning objective is total error, not training fit.

Train/validation/test split logic: fit parameters on train, tune hyperparameters on validation, report performance once on test; leakage occurs when information from validation or test rows enters training (e.g. scaling before splitting).

Cross-validation: k-fold gives k estimates of out-of-sample performance; stratified k-fold preserves class balance for rare events, and time-series data needs rolling-origin or blocked splits to avoid look-ahead bias.

Class imbalance handling: resampling (over/under), class weights, and threshold tuning change the decision boundary but not the underlying data-generating process; report precision, recall, and calibration, not accuracy alone.

Missing data mechanisms: MCAR (missing completely at random), MAR (missing at random given observed variables), MNAR (missing not at random); complete-case analysis is unbiased only under MCAR, while multiple imputation is valid under MAR.

Reproducibility requires recorded data versions, code, seeds, package versions, and environment; without these, a model cannot be independently re-run, which conflicts with actuarial documentation and peer-review expectations.

Pricing and underwriting: GLMs remain the regulatory baseline in many markets; gradient boosting and neural networks are used for risk segmentation but often need explainability surrogates or constrained monotonic relationships.

Claims reserving: machine learning supplements chain-ladder and Bornhuetter-Ferguson by modelling claim-level development; individual claim models can capture case-estimate behaviour and settlement lags that aggregate triangles miss.

Fraud and anomaly detection is a rare-event problem: precision-recall curves and lift charts are more informative than ROC curves, and false-positive cost (customer friction) must be weighed against claim savings.

Mortality and longevity: Lee-Carter and its extensions model log mortality with an age profile and a time index; regularised and machine learning variants forecast the time index or allow richer age-period interactions.

Customer retention and lifetime value: churn models feed survival or hazard models; expected lifetime value = expected duration x expected margin, so both components need separate validation.

Capital and risk modelling: machine learning metamodels approximate slow simulation engines (e.g. nested stochastic), but the surrogate must be validated on out-of-sample scenarios and checked for tail behaviour.

Text and document analytics: natural language processing supports claims triage, complaint classification, and policy wording comparison; performance is judged by precision and recall per class, not aggregate accuracy.

Deployment and monitoring: population stability index and drift tests on input features, plus periodic recalibration, detect degradation; model risk management requires documented ownership, change control, and retirement criteria.

Generalised linear models: link function g(mu) = linear predictor, variance proportional to a power of the mean; Poisson (counts), gamma (severity), and Tweedie (aggregate loss with mass at zero) are standard actuarial families.

Regularisation: ridge adds an L2 penalty and shrinks coefficients, lasso adds an L1 penalty and performs selection, elastic net combines both; penalty strength is chosen by cross-validation, and features should be scaled first.

Decision trees split on impurity reduction (Gini or entropy for classification, variance for regression); single trees are unstable, so ensembles (bagging, random forests, boosting) trade interpretability for accuracy.

Gradient boosting builds additive trees sequentially, each fitting the negative gradient of the loss; key hyperparameters are tree depth, learning rate, number of trees, and row/column subsampling.

Neural networks: layered linear transforms with non-linear activations; training minimises a loss by gradient descent, with dropout and early stopping as regularisers, and they require large data and careful tuning.

Model evaluation: ROC-AUC measures ranking across thresholds, log loss measures calibration and sharpness, and Gini = 2 x AUC - 1; for regression use RMSE, MAE, and deviance residuals.

Calibration: a model is calibrated if predicted probabilities match observed frequencies; check with reliability diagrams, Hosmer-Lemeshow style tests, or by fitting a logistic recalibration on held-out data.

Explainability tools: variable importance, partial dependence plots, and SHAP values attribute predictions to features; SHAP is additive per prediction but correlated features can split credit misleadingly.

Clustering and dimension reduction: k-means minimises within-cluster sum of squares and needs k chosen in advance; PCA projects onto orthogonal components ordered by explained variance and is sensitive to feature scaling.

Actuarial standards require models to be documented, validated, and communicated with appropriate caveats; using a model outside its validated scope is a professional failure regardless of technical accuracy.

Fairness: protected characteristics (e.g. age, sex, disability, ethnicity) are restricted or prohibited rating factors in many jurisdictions; proxy variables can reintroduce discrimination even when the protected field is excluded.

Explainability versus accuracy trade-off: complex models may be justified only where outcomes can be explained to customers, regulators, and the business, or where a documented surrogate explanation is acceptable.

Data ethics: purpose limitation, data minimisation, lawful basis, retention limits, and subject access rights under data protection law; secondary use of data for a new model may need fresh consent or assessment.

Algorithmic accountability: named model owner, independent validation, audit trail of data and code, and a clear escalation route for adverse findings; model risk frameworks grade models by materiality.

Transparency to customers: where automated decisions materially affect individuals, disclosure and, in some regimes, a right to human review or explanation apply; opacity is not a defence.

Uncertainty communication: present ranges and sensitivity, not single point estimates; overstating model precision to decision-makers is a professional integrity issue.

Emerging AI regulation: risk-tiered obligations (prohibited, high-risk, limited-risk uses) impose documentation, human oversight, and monitoring duties; actuarial work in pricing and claims may fall in higher tiers.

Data science pipeline: problem framing, data acquisition, cleaning and validation, exploratory analysis, modelling, validation, deployment, and monitoring; each stage has actuarial controls for reproducibility and auditability.

Data types matter: structured (tabular, relational), semi-structured (JSON, XML, logs), and unstructured (text, images, audio); actuarial models typically assume structured numeric or categorical inputs, so unstructured sources require feature extraction first.

Bias-variance decomposition: expected squared error = bias squared + variance + irreducible noise; increasing model complexity lowers bias and raises variance, so the tuning objective is total error, not training fit.

Train/validation/test split logic: fit parameters on train, tune hyperparameters on validation, report performance once on test; leakage occurs when information from validation or test rows enters training (e.g. scaling before splitting).

Cross-validation: k-fold gives k estimates of out-of-sample performance; stratified k-fold preserves class balance for rare events, and time-series data needs rolling-origin or blocked splits to avoid look-ahead bias.

Class imbalance handling: resampling (over/under), class weights, and threshold tuning change the decision boundary but not the underlying data-generating process; report precision, recall, and calibration, not accuracy alone.

Missing data mechanisms: MCAR (missing completely at random), MAR (missing at random given observed variables), MNAR (missing not at random); complete-case analysis is unbiased only under MCAR, while multiple imputation is valid under MAR.

Reproducibility requires recorded data versions, code, seeds, package versions, and environment; without these, a model cannot be independently re-run, which conflicts with actuarial documentation and peer-review expectations.

Pricing and underwriting: GLMs remain the regulatory baseline in many markets; gradient boosting and neural networks are used for risk segmentation but often need explainability surrogates or constrained monotonic relationships.

Claims reserving: machine learning supplements chain-ladder and Bornhuetter-Ferguson by modelling claim-level development; individual claim models can capture case-estimate behaviour and settlement lags that aggregate triangles miss.

Fraud and anomaly detection is a rare-event problem: precision-recall curves and lift charts are more informative than ROC curves, and false-positive cost (customer friction) must be weighed against claim savings.

Mortality and longevity: Lee-Carter and its extensions model log mortality with an age profile and a time index; regularised and machine learning variants forecast the time index or allow richer age-period interactions.

Customer retention and lifetime value: churn models feed survival or hazard models; expected lifetime value = expected duration x expected margin, so both components need separate validation.

Capital and risk modelling: machine learning metamodels approximate slow simulation engines (e.g. nested stochastic), but the surrogate must be validated on out-of-sample scenarios and checked for tail behaviour.

Text and document analytics: natural language processing supports claims triage, complaint classification, and policy wording comparison; performance is judged by precision and recall per class, not aggregate accuracy.

Deployment and monitoring: population stability index and drift tests on input features, plus periodic recalibration, detect degradation; model risk management requires documented ownership, change control, and retirement criteria.

Generalised linear models: link function g(mu) = linear predictor, variance proportional to a power of the mean; Poisson (counts), gamma (severity), and Tweedie (aggregate loss with mass at zero) are standard actuarial families.

Regularisation: ridge adds an L2 penalty and shrinks coefficients, lasso adds an L1 penalty and performs selection, elastic net combines both; penalty strength is chosen by cross-validation, and features should be scaled first.

Decision trees split on impurity reduction (Gini or entropy for classification, variance for regression); single trees are unstable, so ensembles (bagging, random forests, boosting) trade interpretability for accuracy.

Gradient boosting builds additive trees sequentially, each fitting the negative gradient of the loss; key hyperparameters are tree depth, learning rate, number of trees, and row/column subsampling.

Neural networks: layered linear transforms with non-linear activations; training minimises a loss by gradient descent, with dropout and early stopping as regularisers, and they require large data and careful tuning.

Model evaluation: ROC-AUC measures ranking across thresholds, log loss measures calibration and sharpness, and Gini = 2 x AUC - 1; for regression use RMSE, MAE, and deviance residuals.

Calibration: a model is calibrated if predicted probabilities match observed frequencies; check with reliability diagrams, Hosmer-Lemeshow style tests, or by fitting a logistic recalibration on held-out data.

Explainability tools: variable importance, partial dependence plots, and SHAP values attribute predictions to features; SHAP is additive per prediction but correlated features can split credit misleadingly.

Clustering and dimension reduction: k-means minimises within-cluster sum of squares and needs k chosen in advance; PCA projects onto orthogonal components ordered by explained variance and is sensitive to feature scaling.

Actuarial standards require models to be documented, validated, and communicated with appropriate caveats; using a model outside its validated scope is a professional failure regardless of technical accuracy.

Fairness: protected characteristics (e.g. age, sex, disability, ethnicity) are restricted or prohibited rating factors in many jurisdictions; proxy variables can reintroduce discrimination even when the protected field is excluded.

Explainability versus accuracy trade-off: complex models may be justified only where outcomes can be explained to customers, regulators, and the business, or where a documented surrogate explanation is acceptable.

Data ethics: purpose limitation, data minimisation, lawful basis, retention limits, and subject access rights under data protection law; secondary use of data for a new model may need fresh consent or assessment.

Algorithmic accountability: named model owner, independent validation, audit trail of data and code, and a clear escalation route for adverse findings; model risk frameworks grade models by materiality.

Transparency to customers: where automated decisions materially affect individuals, disclosure and, in some regimes, a right to human review or explanation apply; opacity is not a defence.

Uncertainty communication: present ranges and sensitivity, not single point estimates; overstating model precision to decision-makers is a professional integrity issue.

Emerging AI regulation: risk-tiered obligations (prohibited, high-risk, limited-risk uses) impose documentation, human oversight, and monitoring duties; actuarial work in pricing and claims may fall in higher tiers.

Data science pipeline: problem framing, data acquisition, cleaning and validation, exploratory analysis, modelling, validation, deployment, and monitoring; each stage has actuarial controls for reproducibility and auditability.

Data types matter: structured (tabular, relational), semi-structured (JSON, XML, logs), and unstructured (text, images, audio); actuarial models typically assume structured numeric or categorical inputs, so unstructured sources require feature extraction first.

Bias-variance decomposition: expected squared error = bias squared + variance + irreducible noise; increasing model complexity lowers bias and raises variance, so the tuning objective is total error, not training fit.

Train/validation/test split logic: fit parameters on train, tune hyperparameters on validation, report performance once on test; leakage occurs when information from validation or test rows enters training (e.g. scaling before splitting).

Cross-validation: k-fold gives k estimates of out-of-sample performance; stratified k-fold preserves class balance for rare events, and time-series data needs rolling-origin or blocked splits to avoid look-ahead bias.

Class imbalance handling: resampling (over/under), class weights, and threshold tuning change the decision boundary but not the underlying data-generating process; report precision, recall, and calibration, not accuracy alone.

Missing data mechanisms: MCAR (missing completely at random), MAR (missing at random given observed variables), MNAR (missing not at random); complete-case analysis is unbiased only under MCAR, while multiple imputation is valid under MAR.

Reproducibility requires recorded data versions, code, seeds, package versions, and environment; without these, a model cannot be independently re-run, which conflicts with actuarial documentation and peer-review expectations.

Pricing and underwriting: GLMs remain the regulatory baseline in many markets; gradient boosting and neural networks are used for risk segmentation but often need explainability surrogates or constrained monotonic relationships.

Claims reserving: machine learning supplements chain-ladder and Bornhuetter-Ferguson by modelling claim-level development; individual claim models can capture case-estimate behaviour and settlement lags that aggregate triangles miss.

Fraud and anomaly detection is a rare-event problem: precision-recall curves and lift charts are more informative than ROC curves, and false-positive cost (customer friction) must be weighed against claim savings.

Mortality and longevity: Lee-Carter and its extensions model log mortality with an age profile and a time index; regularised and machine learning variants forecast the time index or allow richer age-period interactions.

Customer retention and lifetime value: churn models feed survival or hazard models; expected lifetime value = expected duration x expected margin, so both components need separate validation.

Capital and risk modelling: machine learning metamodels approximate slow simulation engines (e.g. nested stochastic), but the surrogate must be validated on out-of-sample scenarios and checked for tail behaviour.

Text and document analytics: natural language processing supports claims triage, complaint classification, and policy wording comparison; performance is judged by precision and recall per class, not aggregate accuracy.

Deployment and monitoring: population stability index and drift tests on input features, plus periodic recalibration, detect degradation; model risk management requires documented ownership, change control, and retirement criteria.

Generalised linear models: link function g(mu) = linear predictor, variance proportional to a power of the mean; Poisson (counts), gamma (severity), and Tweedie (aggregate loss with mass at zero) are standard actuarial families.

Regularisation: ridge adds an L2 penalty and shrinks coefficients, lasso adds an L1 penalty and performs selection, elastic net combines both; penalty strength is chosen by cross-validation, and features should be scaled first.

Decision trees split on impurity reduction (Gini or entropy for classification, variance for regression); single trees are unstable, so ensembles (bagging, random forests, boosting) trade interpretability for accuracy.

Gradient boosting builds additive trees sequentially, each fitting the negative gradient of the loss; key hyperparameters are tree depth, learning rate, number of trees, and row/column subsampling.

Neural networks: layered linear transforms with non-linear activations; training minimises a loss by gradient descent, with dropout and early stopping as regularisers, and they require large data and careful tuning.

Model evaluation: ROC-AUC measures ranking across thresholds, log loss measures calibration and sharpness, and Gini = 2 x AUC - 1; for regression use RMSE, MAE, and deviance residuals.

Calibration: a model is calibrated if predicted probabilities match observed frequencies; check with reliability diagrams, Hosmer-Lemeshow style tests, or by fitting a logistic recalibration on held-out data.

Explainability tools: variable importance, partial dependence plots, and SHAP values attribute predictions to features; SHAP is additive per prediction but correlated features can split credit misleadingly.

Clustering and dimension reduction: k-means minimises within-cluster sum of squares and needs k chosen in advance; PCA projects onto orthogonal components ordered by explained variance and is sensitive to feature scaling.

Actuarial standards require models to be documented, validated, and communicated with appropriate caveats; using a model outside its validated scope is a professional failure regardless of technical accuracy.

Fairness: protected characteristics (e.g. age, sex, disability, ethnicity) are restricted or prohibited rating factors in many jurisdictions; proxy variables can reintroduce discrimination even when the protected field is excluded.

Explainability versus accuracy trade-off: complex models may be justified only where outcomes can be explained to customers, regulators, and the business, or where a documented surrogate explanation is acceptable.

Data ethics: purpose limitation, data minimisation, lawful basis, retention limits, and subject access rights under data protection law; secondary use of data for a new model may need fresh consent or assessment.

Algorithmic accountability: named model owner, independent validation, audit trail of data and code, and a clear escalation route for adverse findings; model risk frameworks grade models by materiality.

Transparency to customers: where automated decisions materially affect individuals, disclosure and, in some regimes, a right to human review or explanation apply; opacity is not a defence.

Uncertainty communication: present ranges and sensitivity, not single point estimates; overstating model precision to decision-makers is a professional integrity issue.

Emerging AI regulation: risk-tiered obligations (prohibited, high-risk, limited-risk uses) impose documentation, human oversight, and monitoring duties; actuarial work in pricing and claims may fall in higher tiers.

Data science pipeline: problem framing, data acquisition, cleaning and validation, exploratory analysis, modelling, validation, deployment, and monitoring; each stage has actuarial controls for reproducibility and auditability.

Data types matter: structured (tabular, relational), semi-structured (JSON, XML, logs), and unstructured (text, images, audio); actuarial models typically assume structured numeric or categorical inputs, so unstructured sources require feature extraction first.

Bias-variance decomposition: expected squared error = bias squared + variance + irreducible noise; increasing model complexity lowers bias and raises variance, so the tuning objective is total error, not training fit.

Train/validation/test split logic: fit parameters on train, tune hyperparameters on validation, report performance once on test; leakage occurs when information from validation or test rows enters training (e.g. scaling before splitting).

Cross-validation: k-fold gives k estimates of out-of-sample performance; stratified k-fold preserves class balance for rare events, and time-series data needs rolling-origin or blocked splits to avoid look-ahead bias.

Class imbalance handling: resampling (over/under), class weights, and threshold tuning change the decision boundary but not the underlying data-generating process; report precision, recall, and calibration, not accuracy alone.

Missing data mechanisms: MCAR (missing completely at random), MAR (missing at random given observed variables), MNAR (missing not at random); complete-case analysis is unbiased only under MCAR, while multiple imputation is valid under MAR.

Reproducibility requires recorded data versions, code, seeds, package versions, and environment; without these, a model cannot be independently re-run, which conflicts with actuarial documentation and peer-review expectations.

Pricing and underwriting: GLMs remain the regulatory baseline in many markets; gradient boosting and neural networks are used for risk segmentation but often need explainability surrogates or constrained monotonic relationships.

Claims reserving: machine learning supplements chain-ladder and Bornhuetter-Ferguson by modelling claim-level development; individual claim models can capture case-estimate behaviour and settlement lags that aggregate triangles miss.

Fraud and anomaly detection is a rare-event problem: precision-recall curves and lift charts are more informative than ROC curves, and false-positive cost (customer friction) must be weighed against claim savings.

Mortality and longevity: Lee-Carter and its extensions model log mortality with an age profile and a time index; regularised and machine learning variants forecast the time index or allow richer age-period interactions.

Customer retention and lifetime value: churn models feed survival or hazard models; expected lifetime value = expected duration x expected margin, so both components need separate validation.

Capital and risk modelling: machine learning metamodels approximate slow simulation engines (e.g. nested stochastic), but the surrogate must be validated on out-of-sample scenarios and checked for tail behaviour.

Text and document analytics: natural language processing supports claims triage, complaint classification, and policy wording comparison; performance is judged by precision and recall per class, not aggregate accuracy.

Deployment and monitoring: population stability index and drift tests on input features, plus periodic recalibration, detect degradation; model risk management requires documented ownership, change control, and retirement criteria.

Generalised linear models: link function g(mu) = linear predictor, variance proportional to a power of the mean; Poisson (counts), gamma (severity), and Tweedie (aggregate loss with mass at zero) are standard actuarial families.

Regularisation: ridge adds an L2 penalty and shrinks coefficients, lasso adds an L1 penalty and performs selection, elastic net combines both; penalty strength is chosen by cross-validation, and features should be scaled first.

Decision trees split on impurity reduction (Gini or entropy for classification, variance for regression); single trees are unstable, so ensembles (bagging, random forests, boosting) trade interpretability for accuracy.

Gradient boosting builds additive trees sequentially, each fitting the negative gradient of the loss; key hyperparameters are tree depth, learning rate, number of trees, and row/column subsampling.

Neural networks: layered linear transforms with non-linear activations; training minimises a loss by gradient descent, with dropout and early stopping as regularisers, and they require large data and careful tuning.

Model evaluation: ROC-AUC measures ranking across thresholds, log loss measures calibration and sharpness, and Gini = 2 x AUC - 1; for regression use RMSE, MAE, and deviance residuals.

Calibration: a model is calibrated if predicted probabilities match observed frequencies; check with reliability diagrams, Hosmer-Lemeshow style tests, or by fitting a logistic recalibration on held-out data.

Explainability tools: variable importance, partial dependence plots, and SHAP values attribute predictions to features; SHAP is additive per prediction but correlated features can split credit misleadingly.

Clustering and dimension reduction: k-means minimises within-cluster sum of squares and needs k chosen in advance; PCA projects onto orthogonal components ordered by explained variance and is sensitive to feature scaling.

Actuarial standards require models to be documented, validated, and communicated with appropriate caveats; using a model outside its validated scope is a professional failure regardless of technical accuracy.

Fairness: protected characteristics (e.g. age, sex, disability, ethnicity) are restricted or prohibited rating factors in many jurisdictions; proxy variables can reintroduce discrimination even when the protected field is excluded.

Explainability versus accuracy trade-off: complex models may be justified only where outcomes can be explained to customers, regulators, and the business, or where a documented surrogate explanation is acceptable.

Data ethics: purpose limitation, data minimisation, lawful basis, retention limits, and subject access rights under data protection law; secondary use of data for a new model may need fresh consent or assessment.

Algorithmic accountability: named model owner, independent validation, audit trail of data and code, and a clear escalation route for adverse findings; model risk frameworks grade models by materiality.

Transparency to customers: where automated decisions materially affect individuals, disclosure and, in some regimes, a right to human review or explanation apply; opacity is not a defence.

Uncertainty communication: present ranges and sensitivity, not single point estimates; overstating model precision to decision-makers is a professional integrity issue.

Emerging AI regulation: risk-tiered obligations (prohibited, high-risk, limited-risk uses) impose documentation, human oversight, and monitoring duties; actuarial work in pricing and claims may fall in higher tiers.