AI Triage Software for Intracranial Haemorrhage: Rule 11 and Clinical Evidence
An SME has developed a cloud-based deep learning tool, NeuroFlag, that analyses non-contrast CT head images and places suspected intracranial haemorrhage cases at the top of the radiology worklist. The intended purpose states it is a 'triage aid for trained radiologists; it does not replace primary reading'. It integrates with PACS and is intended for emergency departments. The manufacturer has assigned a UDI-DI and plans to submit for MDR certification. In the pre-registration package, it claims Class IIa under Rule 11, arguing that a radiologist always reads every image and makes the final diagnosis. The clinical evidence is a retrospective multi-centre study of 1,200 scans (AUC 0.94, sensitivity 92%, specificity 88%) and a usability study with 12 radiologists. There is no prospective study, no PMCF plan, and no cybersecurity file beyond a penetration test. The notified body must decide whether to accept the application and what to require.
1. Is the proposed Class IIa classification defensible under Rule 11 and MDCG guidance? State the class you would expect the notified body to require and explain the reasoning.
Read the worked answer
Rule 11 does not turn only on who makes the final diagnosis. It asks whether the software provides information used to take a decision with a diagnostic or therapeutic purpose, and then looks at the impact of that decision. NeuroFlag's output is used to decide the order in which CT scans are read. A false negative can delay treatment of an intracranial haemorrhage; in an emergency setting that delay can cause death or irreversible deterioration. The manufacturer's argument that the radiologist reads every image is a workflow control, not a classification argument. Under MDCG 2019-11 and MDCG 2021-24, the intended purpose and the clinical workflow must be assessed. If the flag is relied upon to prioritise reading and a missed flag can delay diagnosis, Class III is the defensible outcome. If the workflow guarantees that every scan is read within a clinically safe time regardless of the flag, the impact may be serious deterioration, which supports Class IIb. Class IIa is not adequately justified. The notified body should require a formal classification rationale, a workflow risk analysis, and a revised intended purpose before accepting the application.
2. What clinical evidence would you require to support the claimed intended purpose, and why is the retrospective study alone insufficient?
Read the worked answer
The clinical evaluation must demonstrate clinical performance and clinical benefit for the intended use in the emergency workflow, not only stand-alone accuracy. The retrospective study has several limitations: it cannot show that triage changes time to diagnosis or patient outcomes; it may suffer from spectrum bias and selection bias; and it does not test human-AI interaction, alert fatigue, or integration into PACS. The AUC, sensitivity and specificity are useful performance signals, but they do not establish clinical benefit. I would require a prospective clinical performance study or a well-controlled clinical investigation with clinically relevant endpoints, such as time from image acquisition to radiologist interpretation, rate of missed haemorrhage, and downstream management. The study should be conducted in the intended population and workflow, with pre-specified endpoints, a sample size justification, and a plan for handling indeterminate cases. For a Class III claim, the notified body should also consider whether a clinical investigation is necessary under MDR Article 61 and Annex XIV. A PMCF plan must be added to monitor real-world performance and model drift.
3. Identify the pre-registration gaps outside clinical evidence (cybersecurity, UDI, PMCF) and state the consequence for the assessment.
Read the worked answer
Cybersecurity: the software is cloud-based and processes health data. A penetration test alone is not a cybersecurity file. Under MDR Annex I 17.2-17.4 and MDCG 2019-16, the manufacturer needs a threat model, security risk assessment, SBOM, access control, encryption, vulnerability handling, incident response, and update policy. UDI/EUDAMED: the UDI-DI assignment is necessary, but the Basic UDI-DI, EUDAMED device registration and label/package UDI carrier must be consistent with the intended purpose and classification. PMCF: there is no PMCF plan. Annex XIV Part B requires a proactive PMCF plan; for AI, it should include performance monitoring, drift detection, and re-training/update controls. Consequence: the technical documentation is incomplete. The notified body should not accept the application for substantive assessment until these gaps are closed; otherwise any certificate would rest on unresolved safety and performance issues.