The Model Card That Says 'High Accuracy'
A hospital's innovation lab builds a model that predicts 30-day readmission risk to help discharge planners prioritize follow-up. The model card states 'AUC 0.84, high accuracy' and lists training data as 'electronic health records, 2019–2021.' The lab wants to deploy it in three hospitals with different patient populations and EHR systems. A clinician asks whether the model was validated on patients like hers, and whether it accounts for social determinants of health. The lab lead says the AUC is strong and that 'the model is general.' There is no monitoring plan, and the model will be embedded in the discharge workflow with no override documentation.
1. Identify the gaps in the model card and validation evidence relative to the intended deployment, and explain the risks of the 'it is general' claim.
Read the worked answer
The model card is inadequate for the stated use. Gaps:
- Population validity. Training data 2019–2021 from one setting may not represent the three target hospitals' demographics, case mix, or care pathways. AUC 0.84 on the development distribution says nothing about external validity; performance typically degrades under distribution shift.
- Temporal validity. 2019–2021 spans COVID-era disruptions; readmission patterns and coding changed. The model may encode era-specific artifacts.
- Missing subgroup analysis. No calibration or performance by age, sex, race/ethnicity, language, or social determinants. A high AUC can coexist with poor calibration for subgroups - critical because discharge planners act on predicted risk thresholds, not ranks.
- No feature/SDH documentation. The clinician's question about social determinants is unanswered; if SDH proxies are used, their effect and fairness implications are undocumented.
- No intended-use or limitation statement, no monitoring plan, and no override documentation - all governance failures for a clinical decision-support tool.
The 'it is general' claim is unsupported: generalization is an empirical property, not an assumption. It must be demonstrated by external validation at each site and by subgroup calibration analysis.
2. Design a site-specific validation and deployment plan, including monitoring and override documentation, that would satisfy a clinical governance committee.
Read the worked answer
Pre-deployment validation (per site): Run a silent-mode (shadow) evaluation on each hospital's recent data (e.g., 6–12 months) before any workflow use. Report discrimination (AUC), calibration (calibration slope/intercept, reliability curves), and subgroup performance (age, sex, race/ethnicity, language, payer, SDH indicators). Predefine acceptable thresholds (e.g., calibration slope 0.8–1.2, no subgroup AUC gap > 0.05) and a stopping rule if not met.
Local adaptation: If calibration is off, recalibrate (e.g., Platt scaling) on site data rather than retraining from scratch; document the change. Confirm feature availability and meaning across EHR systems (coding differences can silently break features).
Workflow integration: Present risk as a calibrated probability with a recommended action band, not a bare score. Require clinicians to record whether they followed or overrode the recommendation, with a reason - this is the override documentation.
Monitoring: Track monthly calibration drift, alert on shifts in input distributions (population stability index), and log outcomes (actual readmissions) to detect degradation. Define a review cadence and an owner.
Governance: Update the model card with intended use, limitations, subgroup results, and monitoring plan; obtain committee sign-off; schedule re-validation when populations, coding, or the model change.