Kestrel Health: Architecture for Governing Models Outside a Single Platform, With EU Residency
Kestrel Health runs clinical and commercial AI across four environments: imaging models on AWS SageMaker, a patient-messaging assistant on Azure OpenAI, several forecasting models on watsonx.ai, and two legacy on-premises statistical models that cannot move. Its EU operations generate prompt and response logs that under internal policy and GDPR practice must remain in the EU, while the US team wants a single global dashboard. The CISO will not approve sending raw patient text across regions. Kestrel needs: an inventory and factsheet per model and use case, drift and fairness monitoring for the imaging and forecasting models, evaluation of the messaging assistant for harmful or fabricated output, and an approval workflow with named clinical sign-off. The platform team has asked for a reference architecture, a clear statement of what the governance layer does not do, and a deployment recommendation.
1. Describe the reference architecture and the data flows that make it work across four model environments.
Read the worked answer
Separate the governance plane from the model plane. Kestrel will not consolidate model hosting, so the architecture is a metadata and evidence hub with connectors, not a control point in the inference path.
Components:
- Inventory and use-case register - one record per AI use case and per model, carrying owner, purpose, risk tier, jurisdictions and clinical sign-off requirement.
- Factsheets attached to those records, populated from model metadata where connectors exist and manually where they do not (the on-premises models).
- Connectors and collectors - SageMaker endpoint and monitoring metadata; Azure OpenAI configuration plus sampled prompt/response telemetry; watsonx.ai model metadata; an on-premises collector pushing batch metrics outbound so the models themselves never need inbound access.
- Evaluation and monitoring services - drift, performance and fairness for structured models; quality, toxicity, PII and groundedness evaluation for the messaging assistant, run on sampled data.
- Workflow and evidence store - validation tasks, clinical approvals, findings, remediation and change history, all attributable and timestamped.
- Policy and obligation mapping with dashboards, role-based, so the US team sees global aggregates while EU users see EU records.
Critical data flow: evaluation of the messaging assistant requires prompt and response content. Run the evaluation and the log store in the EU region for EU traffic; send only derived metrics and de-identified aggregates to the global dashboard. That satisfies the CISO without breaking the single-view requirement.
Latency point: monitoring and evaluation are asynchronous. Nothing in this architecture blocks a live inference call.
2. What does the governance layer not do, and why does stating that matter for this deployment?
Read the worked answer
It does not sit in the runtime request path. If the messaging assistant produces a harmful response, the governance platform records and evaluates it after the fact. Blocking, redaction and refusal must be implemented in the application, an API gateway, or the model provider's guardrails. Kestrel's clinical safety case cannot rest on monitoring dashboards.
It does not decide risk tier or clinical acceptability. A clinical sign-off is a professional judgement. The platform routes and records it; it cannot substitute for a clinician.
It does not automatically discover every model. Shadow deployments in a team's own Azure subscription will be invisible until procurement, cloud spend or identity logs are reconciled. Kestrel must run that reconciliation as a process.
It does not make a vendor model transparent. Azure OpenAI evaluation gives evidence of behaviour on Kestrel's samples, not the provider's training data or full intended-purpose documentation.
It does not fix data quality. Fairness results are only as good as the labels and demographic data feeding them.
It does not make the on-premises models self-documenting. Two models require manual factsheet entry and a human process to keep them current.
Why say this out loud: the failure mode in a four-environment estate is that a governance program is announced, the dashboard goes green, and everyone assumes safety controls exist at runtime. Naming the boundary now sets expectations, forces the guardrail work into the application roadmap, and protects the program's credibility when the first incident is reviewed.
3. Recommend a deployment model. What factors decide it, and what would change your answer?
Read the worked answer
Recommendation: a region-scoped deployment with EU evaluation and log storage inside the EU, feeding a global aggregated view. In practice this means confirming current regional availability for the SaaS offering and, if EU residency or key-management requirements cannot be met that way, deploying the self-managed Cloud Pak for Data form in Kestrel's EU environment. Verify entitlement and regional availability at engagement time; do not assert it from memory.
Deciding factors:
- Residency and lawful transfer. Where do prompt and response logs, factsheets and evaluation results physically sit, and who can administer them? This is the binding constraint, not price.
- Data sensitivity and keys. Whether Kestrel needs customer-managed keys and whether the SaaS tenancy supports them.
- Control of upgrade cadence. A clinical environment has change-control obligations; a managed service's release cadence must fit them.
- Connectivity to four model planes, including an outbound-only path for on-premises models.
- Operational capacity. Self-managed shifts patching and availability onto the platform team; with two legacy models and a small team, that cost is real.
- Total cost over three years including the operational burden of self-managed.
What would change the answer: if EU residency for log content could be satisfied by the SaaS offering with in-region processing and administrative access controls, self-managed is hard to justify. Conversely, if a regulator or the CISO requires that no telemetry leaves Kestrel-controlled infrastructure, SaaS is out regardless of convenience. Get that requirement in writing before architecture is finalised.