Contractor access and a leaked CI key after an audit

Containment should stop further misuse first, then preserve evidence, then restore controlled operations.

1. Revoke the leaked CI key and the departed contractor's IAM user. Do this before rotating everything else, because the key is public and the contractor account is known-active. Record the exact time and the resources each identity could reach.
2. Disable the shared administrator account and replace it with named emergency access. A shared account cannot provide attribution and is too broad to leave active during an incident.
3. Preserve logs before they expire. Export audit, API, and access logs for the period since the key was committed. If logs show which identity created resources, tag or isolate those resources for investigation. Do not delete unknown resources until ownership is established.
4. Rotate other credentials that may have been exposed in the same repository or CI system, including storage keys and any deployment tokens. Assume the repository leak exposed more than one secret.
5. Restrict outbound use of the old key by disabling the CI pipeline's ability to deploy until a new, scoped credential is in place. Replace it with a short-lived or narrowly scoped deployment identity, not another long-lived administrator key.
6. Enable MFA for all remaining human users, starting with anyone who can change IAM, billing, or production resources.

The order matters: revoke public secrets and active stale identities first, preserve evidence second, and restore operations with narrower credentials third.

Use named identities, role-based groups, and explicit boundaries rather than a shared administrator account.

• Human identities: Every engineer and contractor gets a named IAM user or federated identity. No shared accounts. MFA is required for all human access.
• Contractor group: Permissions limited to test resources, such as a test private network, test compute, and test object storage prefixes. Deny production compute, production storage, billing, and IAM changes. If contractors need to read production logs for support, grant read-only access to a specific log prefix rather than broad production access.
• Full-time engineer group: Can deploy and operate production compute, storage, and networking, but cannot change billing settings or IAM policies. Production changes should go through a deployment role, not direct console administrator rights.
• Platform/security administrators: A small separate group with IAM and billing rights, protected by MFA and ideally by a break-glass process. These rights should not be part of the normal engineer role.
• CI deployment identity: A dedicated non-human identity with only the permissions needed to create and update the specific resource types in the pipeline. Scope it by resource group, private network, and storage prefix. Do not reuse a human administrator credential.
• Resource boundaries: Separate production and test into different private networks or accounts/projects where possible. Contractors should not be able to reach production subnets even if they guess a resource name.

The key judgement is that job function, environment, and action type are separate dimensions. A contractor who supports testing should not inherit production deploy rights merely because they are an engineer.

Attribution requires that every action be tied to a named identity and that resource creation be logged and tagged.

• Eliminate shared credentials. A shared administrator account makes attribution impossible by design. Named users and a distinct CI identity ensure that future API calls identify the actor.
• Enable and centralize audit logging. Send control-plane and API logs to a separate, append-only log destination that the compromised identity cannot modify. Include the identity, source address, timestamp, action, and resource identifier.
• Require tags at creation. Enforce tags for owner, team, environment, and purpose. Where the platform supports it, use a policy that denies resource creation without the required tags. This makes orphaned resources easier to find and assign.
• Inventory and reconcile. Compare the resource inventory against the deployment pipeline's expected state. Resources that do not match a known stack or tag should be quarantined and investigated, not deleted blindly.
• Alert on anomalous creation. Trigger alerts for resource creation outside the CI pipeline's normal pattern, especially from a new source address or at an unusual time.
• Review IAM and billing changes separately. These are high-impact actions and should generate alerts to the security team, not just sit in logs.

The design principle is that containment stops the incident, but attribution prevents the next one from being unmanageable. Logs must be tamper-resistant and identities must be unique.