I85PREP
⌘ K
AI-200 study materials
AI-200 study resources

Northwind Capital: hardening a shared Foundry and Search platform

AI-200 · Put your knowledge to work in context.

All case studies

Northwind Capital: hardening a shared Foundry and Search platform

Northwind Capital runs four regulated document workflows on Azure AI. A platform team owns one Foundry (AI Services) resource in West Europe and one shared Azure AI Search service; 40 developers across three feature teams build against them. Today every app's configuration holds the same Search admin key and the same Azure OpenAI key, model deployments are created ad hoc (14 deployments across 6 models), and the monthly AI budget has overrun three months running. Security added two requirements: no public network access to any AI resource, and each team's data readable only by that team's workload. Finance wants chargeback per team. Engineering also reports that a single dev load test saturated the shared deployment and made production slow for 40 minutes.

1. Specify the authentication and network changes needed to meet Security's two requirements, and name the first thing that will break in each team's delivery pipeline.

Read the worked answer

Authentication - replace keys with Entra ID

  • Give each workload its own managed identity (or workload identity federation for CI runners) rather than sharing keys. Keys give no per-team attribution and rotation breaks all 40 developers at once.
  • Assign data-plane roles at the narrowest scope: Cognitive Services OpenAI User on the model resource, Search Index Data Reader for query-only apps, Search Index Data Contributor only for the nightly indexing job, Search Service Contributor for platform admins.
  • Set disableLocalAuth = true on both the AI Services account and the Search service. Otherwise the keys remain a working bypass and the network work is pointless.

Network

  • Private endpoint per resource, publicNetworkAccess = Disabled, private DNS zones (privatelink.openai.azure.com, privatelink.cognitiveservices.azure.com, privatelink.search.windows.net) linked to every VNet that must reach them.
  • Teams run in spoke VNets peered to the hub; build agents move into the VNet (self-hosted runner or hosted runner with VNet injection).

What breaks first

  1. Developer laptops: the public FQDN no longer resolves once public access is disabled - DNS resolution fails before auth is even attempted.
  2. CI pipelines: the identity exists but has no role assignment, so token acquisition succeeds and the first call returns 403.
  3. Apps constructed with AzureKeyCredential fail immediately once local auth is off.

If a team needs public access for a proof of concept, give them a separate sandbox resource in an unregulated subscription - never relax the shared production resource.

2. The shared deployment causes noisy-neighbour incidents and Finance cannot attribute spend. Design the deployment and capacity model, and explain how chargeback should work.

Read the worked answer

Diagnose first: one deployment is one capacity bucket and one quota. A dev load test and production traffic consume the same tokens-per-minute budget, so the incident is a design fault, not a discipline problem.

Split by environment and team

  • Separate deployments per team per environment, each with its own quota headroom.
  • Use standard (pay-per-token) deployments for dev/test - they burst without pre-commitment.
  • Reserve provisioned throughput only for the production deployment whose peak is predictable: it is billed for the reserved capacity whether or not traffic arrives, so it is only cheaper above a sustained utilisation floor. Keep a standard deployment behind it as spillover for bursts.
  • Note quota is allocated per subscription, per region, per model; 14 deployments of the same model do not create 14× capacity. Plan a secondary region pair for failover.

Measure, don't guess Track provisioned utilisation, token counts, and latency per deployment in Azure Monitor; the utilisation metric is what tells you whether PTU was the right purchase.

Chargeback

  • Tag each deployment's parent resource with team and environment; Cost Management gives per-team spend for dev/test.
  • For per-request attribution on a shared production deployment, front it with API Management and emit token usage as a metric per subscribing identity. Token counts, not call counts, are the unit that correlates with cost.

What would change the answer: if all teams had flat, low, unpredictable traffic, per-team standard deployments plus a hard quota cap would be simpler than PTU.

3. The shared Search service holds all four workflows' content. How do you satisfy 'only that team's workload can read that team's data'?

Read the worked answer

Two candidate designs

  1. Shared index with a mandatory filter on teamId - cheapest, but isolation now depends on every query path in every app remembering the filter. One forgotten filter, one debug script, one exported query is a cross-team data leak, which is exactly the failure mode Security is worried about.
  2. Index per team in the shared service - recommended. Isolation is enforced by the index boundary, not by query discipline. Pair it with an identity that only has Search Index Data Reader on its own index (index-level role assignment), so even a bug cannot reach another team's index.

When to go further: if the four workflows are in different regulatory scopes, separate Search services (or separate resource groups in different subscriptions) give a cleaner audit story at the cost of duplicated platform overhead. Recommend that only for the workflow with the strictest scope.

Implementation notes

  • Even with per-team indexes, add a defence-in-depth teamId filter for the indexing job, since documents arrive from a shared landing zone.
  • Keep admin/query keys disabled; the isolation argument collapses if one shared admin key exists.
  • Make the choice explicit in the platform docs: index-boundary isolation is a design property, filtering isolation is a coding convention.