190 System Admins, a shared CI key, and a policy change nobody saw

Why System Admin is the wrong default. In Prisma Cloud, permissions are the combination of role (what actions) and scope (which enterprise group / account groups). System Admin sits at the top of that stack: it administers users, roles, cloud account onboarding, and policies across the tenant. The 190 users who "just need to see alerts" can therefore disable policies, onboard accounts, and change other people's access - which is precisely how last month's problem happened. Over-privilege also destroys attribution: if everyone is an admin, an audit log entry tells you the action, not the intent or the accountability.

Persona mapping:

• MSSP contractor, 90 days. Narrowest read-only role available (account-group-scoped read-only or a custom role with view-only permission sets), scoped to exactly the three account groups, with an explicit end date in the ticket and a calendar reminder to disable. Time-boxed scope, not time-boxed trust.
• Auditor. Report and compliance read permissions with export, no policy write, no account onboarding, no user management. If the only option that can export is a broad role, that is a product limitation to raise - do not "temporarily" hand over policy admin for a quarterly task.
• Jenkins. Not a person, not System Admin - a dedicated service account with a role limited to scanning/pushing results and reading policies (see next task).
• The remaining 190. Re-derive from job function: SOC analyst (alert and investigate), cloud platform (onboarding and config), compliance (read), a small policy-admin group (single digits). Expect the count to fall by an order of magnitude.

SAML group mapping. Map Okta groups to Prisma Cloud roles and scopes so the IdP is the source of truth: joining a group grants access, leaving removes it. The deprovisioning story becomes "Okta deactivates the account", which actually happens when someone leaves, unlike a quarterly console review. Test the mapping with one pilot group before enforcing, and keep one break-glass local account.

Split the account by pipeline group, not one key for everything. Create service accounts per group of pipelines (for example ci-images, ci-iac) so the audit log says which system did what and so a leaked key has a bounded blast radius. One shared key across 22 pipelines gives you neither.

Scope. The role needs to submit scan requests, read the results, and read the policies that gate the build. It must not modify policies, onboard cloud accounts, or manage users. If the CI scanner only needs to push results and query, a custom role built from those permission sets is better than the nearest built-in role - validate with a dry run that the pipelines still pass.

Key handling. Stop storing the key in a Jenkins credential that every administrator can read. Options in order: fetch it at runtime from a secrets manager (Vault, AWS Secrets Manager) using the agent's own workload identity, or use a Jenkins folder-scoped credential visible only to the pipeline folder. Never in a Jenkinsfile, never in a Docker image, never in a repo - and add a pre-commit/secret scan to catch the next person who tries.

Rotation. Assume two active keys so rotation is non-disruptive: create the new key, deploy it, verify pipelines, revoke the old one. Rotate on a 90-day schedule and immediately on any suspicion. Prisma Cloud access keys do not expire on their own, so the schedule has to live somewhere real - a calendar event owned by a named person is better than a policy document.

Detecting misuse. Forward Prisma Cloud audit logs to the SIEM and alert on: API calls from IP ranges outside the CI egress (add an allowlist where supported), calls to endpoints the CI role never uses (policy writes, user management), a spike in volume, and any authentication failure burst. Review the service account's last-used time quarterly and retire unused accounts - the most dangerous credential is a forgotten one that still works.

Accept that people will change policy; design so the change is visible and reviewed. The failure last month was not that an engineer had the ability, it was that nobody knew for three weeks.

1. Shrink the policy-write group. Restrict policy administration to a named group (single digits) and give everyone else read. Every additional holder of policy admin is another person who can create this incident, and there is no operational need for 190.

2. Ship audit logs to somewhere durable. Forward Prisma Cloud audit logs to the SIEM via the integration, so the record survives the tenant and can be correlated with CI deployments and HR events. Retaining them only inside the console makes review optional and tampering easy.

3. Alert on the specific events. SIEM rules for: policy disabled or deleted, policy severity lowered, policy RQL edited, alert rule or integration deleted, user role elevated, service account key created. These are low-volume events - noise is not an excuse for not alerting on them.

4. Reconcile desired state, don't trust it. Keep a versioned list of policies that must be enabled at a given severity (exported from the API), and run a scheduled job that diffs live state against it, opening a ticket on drift. This catches the change even if the audit log pipeline is down, and it catches the case where someone disables a policy and nobody was watching the SIEM.

5. Require a reason, and make silence expensive. A pre-release "clear the noise" action should go through a ticket with a linked exception and an expiry, not an ad-hoc console click. Pair that with a weekly report of enabled-policy delta and route it to the security lead - a change with no ticket becomes a conversation within seven days, not three weeks.

6. Fix the underlying incentive. The engineer disabled 12 noisy policies because they were blocking a release. Give them a legitimate path - scoped exceptions with expiry, or a policy tuning request - or they will keep finding the illegitimate one.