Publishing an AI sepsis model trained on patient data

I would treat the model and the API as potential disclosure channels, not as anonymised artefacts. Model weights can memorise rare patterns, especially with genomics, small ethnic subgroups and free-text mental-health notes. An attacker could use membership inference to test whether a specific patient was in the training set, or model inversion to reconstruct features. The API creates additional risks: repeated queries can probe subgroup behaviour, and postcode-level residence plus ethnicity can make small groups identifiable. I would assess whether the original consent covered research use, model training, publication of weights, commercial reuse and overseas processing. Consent that covered treatment or unspecified research may not cover a public API or a commercial licence. Under the PDPO, I would examine purpose limitation, data minimisation and security, and I would ask whether the ethics approval and patient information were clear enough. The 0.91 AUC is not evidence of privacy safety; a good model can still leak.

I would separate the release of weights from the release of an API. Releasing weights is effectively irreversible, so it should require a stronger assessment and probably a controlled research licence rather than an open public download. For the API, I would require query limits, rate limiting, anomaly detection, no raw-record return, and aggregation or suppression for small subgroups. Differential privacy or a privacy budget could be applied to outputs, with a documented epsilon and utility trade-off. I would remove direct identifiers and quasi-identifiers from training where possible, and test membership inference and attribute inference before release. A model card should state the training population, intended use, known limitations and prohibited uses. External validation should be done through a trusted research environment or federated evaluation before broad access. The goal is to enable validation without turning the API into a reconstruction tool.

I would treat free hosting as a data-governance transaction, not a cost saving. The vendor is asking for usage data, which may include query patterns, hospital identifiers, patient characteristics or model behaviour. That is a new purpose and possibly a new disclosure, so it needs a lawful basis, ethics review and contractual limits. I would ask what data is collected, whether it can be aggregated, how long it is retained, who can access it, whether it is used to train the vendor's models, and whether the university retains control. I would also assess exit and continuity: a free service can create lock-in, and the HKD 3.8 million cloud bill may reappear later. If the offer is accepted, the contract should prohibit re-identification, restrict secondary use, require deletion on exit and give the university audit rights. I would not accept the offer simply because it reduces the current bill; the governance question is whether the usage data is a fair and lawful exchange for the hosting.