Tailspin: 38 Client Secrets, a GitHub Pipeline, and Sign-Ins from 12 Countries
Tailspin operates a multi-tenant SaaS product on Azure. The tenant holds 140 app registrations; 38 authenticate with client secrets, and 11 expired in the last quarter, causing three production outages. Deployment pipelines in GitHub Actions authenticate with a client secret stored as a GitHub secret and hold Contributor on the production subscription. An Azure Function reads secrets from Key Vault using a credential in its application settings. The security team sees 4,000 service principal sign-ins per day originating from 12 countries. A penetration test found 22 app registrations with tenant-wide mail and file permissions granted by a single admin consent. Tailspin holds 50 Microsoft Entra Workload ID Premium licenses.
1. Eliminate the stored secrets for the GitHub pipeline and the Azure Function. Justify why the same mechanism does not fit both.
Read the worked answer
Classify each credential holder by where its code runs.
- Azure Function โ managed identity. It runs on an Azure resource, so give it a system-assigned managed identity, grant that identity Key Vault Secrets User via Azure RBAC, and delete the credential from application settings. There is now no secret to expire or leak.
- GitHub Actions โ workload identity federation. The runner is not an Azure resource, so there is no managed identity to assign. Instead add a federated credential to the app registration with issuer = GitHub's OIDC endpoint, audience
api://AzureADTokenExchange, and a subject restricted to a specific repo and environment (repo:org/repo:environment:prod). The workflow then requests a short-lived token per run. Delete the GitHub secret afterwards - leaving it in place defeats the change.
Why not the same mechanism: a managed identity exists only inside Azure's trust boundary; a federated credential trusts an external OIDC issuer. Choosing the wrong one produces either 'identity not found' or an over-broad subject claim.
The 38 secrets: the multi-tenant SaaS app still needs a credential to authenticate to customer tenants and to your own APIs. There, federation does not apply - rotate to certificates stored in Key Vault or Managed HSM, configure credential preference so certificate wins when both exist, and set a hard rule: no secret with a lifetime over six months. Aim to drive the secret count to zero rather than to manage it better.
2. Three outages came from expired secrets even though expiry notification emails are enabled. What monitoring do you build instead?
Read the worked answer
Treat notification email as a backup, not a control. Expiry mail only reaches the addresses the platform has, often a distribution list nobody owns, and it does not create an accountable owner per application.
Build a scheduled expiry report:
- A job (Azure Automation runbook or Function) with a system-assigned managed identity holding
Application.Read.Allqueries Graph for/applications?$select=appId,displayName,passwordCredentials,keyCredentialsand for the corresponding service principals. - Compute days-to-expiry per credential and emit a table: application, credential type, end date, owner, environment.
- Alert at 60 / 30 / 7 days into the owning team's channel, not into a mailbox, and open a work item at 30 days. Escalate to the platform team at 7 days.
- Require an owner on every app registration and on the associated service principal; an application without an owner is itself a finding.
The strategic fix matters more than the alert: three outages from the same cause means the rotation process is manual and unowned. Prioritise converting the production-critical apps to federation or managed identity so those credentials stop existing, and reserve certificate rotation for the small residual set.
3. Given 50 Workload ID Premium licenses against 140 app identities, how do you reduce the risk from 4,000 service principal sign-ins across 12 countries and the 22 over-permissioned apps?
Read the worked answer
You cannot cover all 140, so make a defensible prioritisation.
- Attribute the sign-ins before you react. Filter service principal sign-in logs by credential type and IP, and separate Microsoft first-party traffic from your own identities. Twelve countries may be three real identities plus CDN and Microsoft noise.
- Rank by blast radius, not by volume. The 22 apps with tenant-wide mail and file permissions are the top risk regardless of where they sign in from. Remediate those first: replace tenant-wide Graph application permissions with narrower scopes where the workload permits, and apply application-level scoping (for example an application access policy restricting which mailboxes an app may touch) instead of tenant-wide mailbox access. Re-consent under an approval process - a single admin consent granting tenant-wide mail is a process failure, not just a configuration error.
- Spend the 50 licenses on the highest-blast-radius identities. Confirm what the license actually covers in your agreement before scoping - you are buying coverage for a subset of 140, not for all of them. Apply Conditional Access for workload identities where the supported conditions match your risk (location and risk conditions; the grant surface is narrower than for users - validate the supported controls rather than assuming user-policy parity).
- Managed identities are not covered by workload identity Conditional Access. Protect those with network controls: Key Vault firewall and private endpoints, and constrained egress.
- For everything unlicensed, the fallback is credential hygiene: short-lived certificates, federated credentials where possible, and Sentinel detections on anomalous service principal sign-in location.