The Robo-Advisor That Passed Its Own Test
Crestline Wealth deploys an AI robo-advisor to 40,000 retail clients. The model was validated on a held-out dataset drawn from the same 2019–2022 period as its training data, achieving 91% accuracy on suitability classification. In production, the model recommends portfolios by predicting each client's risk tolerance from transaction patterns and app usage. A routine audit finds that clients over 65 receive conservative portfolios at 2.4x the rate of younger clients with identical stated risk scores and identical financial profiles. The audit also finds that the model's 'explanation' feature cites the client's stated risk score as the primary driver in 78% of cases, even when the actual recommendation was driven by transaction-pattern features. Crestline's compliance team asks whether the model can continue in production while a remediation plan is drafted. Separately, a client who received a conservative recommendation and missed a 14% equity rally has complained.
1. Evaluate the model's validation methodology and the age disparity. Is the model fit for production?
Read the worked answer
Validation flaw. The held-out set was drawn from the same period as training data. That is not an out-of-sample test in any meaningful sense - it tests interpolation, not generalization. A 91% figure from such a split is inflated and tells you almost nothing about performance on new clients or changed market conditions. Proper validation requires a genuinely later time period (walk-forward), and ideally a geographic or product holdout.
Age disparity. 2.4x the rate of conservative recommendations for over-65s with identical stated risk scores and financial profiles is a disparate impact signal. Two explanations are possible:
- The transaction-pattern features are proxies for age (e.g., pension deposits, lower trading frequency, fixed-income holdings). The model has learned age through a proxy - a classic proxy-discrimination failure.
- The model is correctly detecting something the stated risk score misses. This is possible but must be demonstrated, not assumed.
Fit for production? No. The combination of an invalid validation methodology and an unexplained protected-class disparity means the model's outputs cannot be defended to a regulator or a client. Suspend new recommendations or restrict the model to a supervised mode, and investigate the feature-level contribution to the age gap.
Key test. Retrain without transaction-pattern features and measure whether the age disparity persists. If it disappears, the features were proxies. If it persists, the stated risk score itself encodes age effects.
2. The explanation feature is misleading in 78% of cases. Assess the regulatory and client-harm implications.
Read the worked answer
This is the most serious finding. A misleading explanation is worse than no explanation, because it defeats the client's and the regulator's ability to detect the real driver. If the model says 'we recommended this because you told us you were conservative' while actually relying on app-usage patterns, then:
- The client's informed consent is invalid - they consented to a rationale that was not operative.
- Suitability records are inaccurate, which is a record-keeping breach independent of the advice quality.
- Any complaint or ombudsman review will rely on the stated reason and reach the wrong conclusion.
Likely cause. The explanation layer is a post-hoc surrogate (e.g., a simple model fitted to approximate the complex one), and it has latched onto the stated risk score because that feature is stable and human-legible. It is explaining plausibility, not causality.
Required remediation.
- Replace surrogate explanations with methods that faithfully reflect the actual model's feature contributions, and validate them against ground truth by perturbation testing.
- Where faithful explanation is not technically achievable, do not deploy the model - an unexplainable suitability decision is not defensible.
- Re-review all 40,000 recommendations made under the misleading explanation, prioritizing clients whose stated and actual drivers diverged.
Regulatory posture. Self-identify the explanation defect to the regulator with a remediation timeline. Discovery by the regulator converts a fixable control failure into a trust failure.
3. Advise on the complaining client and on whether to keep the model live during remediation.
Read the worked answer
The complaining client. The client missed a 14% equity rally after receiving a conservative recommendation. Redress depends on causation:
- If the recommendation was suitable on the client's actual circumstances, there is no loss to compensate - market underperformance relative to a benchmark the client was never promised is not a loss.
- If the recommendation was driven by a proxy feature rather than the client's stated preferences, the advice was not what the client asked for, and redress is arguable.
Practical approach. Review the individual file: stated risk score, actual feature contributions, and whether the client was told the basis of the recommendation. If the stated score would have produced a balanced portfolio and the model overrode it on transaction patterns, offer to rebalance at no cost and consider compensation for the differential. Do not offer blanket compensation to all 40,000 - that admits a loss that may not exist and creates a moral hazard.
Keep the model live? No, not in its current form. Recommended path:
- Pause new automated recommendations for clients where the age disparity is material, or route them to human review.
- Keep existing portfolios in place - forced liquidation would crystallize losses and is not required by the defect.
- Fix the explanation layer first, because it is the control that lets you detect everything else.
- Re-validate on a genuine out-of-time sample before restoring full automation.
Board-level point. The question 'can we keep it live?' is really 'what is our risk appetite for undisclosed proxy discrimination?' That is a board decision, documented, not a compliance-team decision.