Eight Live Keys, Sixty Repositories, Eight Weeks to Audit
A fintech has 60 repositories across 15 teams. A central security team runs a secret scanner and an SBOM/vulnerability scanner in CI. Last quarter, 8 repositories contained live cloud keys; 3 were in commit history, 5 in test fixtures. The scanner blocks merges when it finds a secret, but teams can request an admin override. Over 6 months, 22 overrides were used, and 17 were never remediated. The security team wants to enforce a hard block with no overrides and add a policy that fails builds for any critical vulnerability in a dependency. The engineering teams complain that the scanner has a 9% false positive rate, scans take 11 minutes per repository, and critical vulnerabilities often have no fix available. The company is preparing for a SOC 2 audit in 8 weeks and must show timely remediation.
1. How would you respond to the 8 leaked keys, and what evidence would you preserve for the audit?
Read the worked answer
Treat the keys as compromised, not merely exposed. Rotate or revoke every live key first, then determine scope from provider audit logs and billing or API activity. Deleting the secret from Git history does not make it safe; rotation must come first. For the three keys in commit history, coordinate a history rewrite with repository owners, force-push only after all clones and forks are accounted for, and invalidate CI caches. For the five in test fixtures, replace them with obviously fake values and add pre-commit and CI checks that reject key-like patterns.
Preserve evidence: scanner findings with repository, file, commit SHA, and key identifier; provider audit logs showing use; rotation and revocation timestamps; incident tickets; approval records; notification records; and the post-remediation scan showing the secret is gone. Record why each key existed and which control failed. For the audit, the important story is timely detection, containment, and prevention, not that no secret ever appeared.
After containment, run a bounded search for other credential patterns and review CI logs for accidental printing. Do not paste the live secret into a ticket or chat; use a fingerprint or provider key ID instead.
2. Evaluate the proposal for a hard block with no overrides and failing any critical vulnerability. What policy would you recommend that satisfies audit and engineering constraints?
Read the worked answer
A hard block with no overrides sounds strong but often produces shadow bypasses or blocks urgent fixes. For secrets, make the default a hard block in pre-commit and CI, but allow a break-glass exception only with security approval, a time limit, a compensating control, and automatic rotation. No admin merge without an audit record. The exception should be narrow and expire; the goal is to make the safe path easier, not to create an unreviewable permanent bypass.
For vulnerabilities, failing on any critical dependency is too blunt when no fix exists. Gate on exploitability and reachability where possible, and on a risk-based SLA. Fail builds for critical vulnerabilities that are reachable and have a fix, or when the package is in a deployed runtime path. For critical findings without a fix, require a documented risk acceptance with an owner, compensating control, review date, and a tracking issue. Use VEX or equivalent exploitability metadata if the team can produce it. Tune the 9% false positive rate with baselines and expiring allowlists rather than permanent suppressions. Reduce the 11-minute scan time with dependency caching, incremental scans, and parallel jobs.
This satisfies audit expectations because it shows a consistent policy, documented exceptions, and timely remediation rather than an unattainable zero-failure rule.
3. What metrics and operating model would show the program is improving before the audit?
Read the worked answer
Measure the things the audit will actually sample: time from secret exposure to revocation, percentage of repositories with scanning and branch protection, number and age of open exceptions, percentage of critical vulnerabilities past SLA, and evidence completeness. Also track false positive rate, scan p95 duration, override count, and the number of findings remediated versus risk-accepted. A falling finding count alone can hide unremediated risk if teams simply stop scanning or suppress results.
Operate the program through security champions in each team, a paved-road pipeline that runs scanning by default, pre-commit hooks for secrets, and a weekly review of exceptions and critical findings. Publish a clear remediation SLA and an escalation path. Before the audit, prioritize live secrets, exploitable critical vulnerabilities, and missing evidence over cosmetic backlog reduction. Run a tabletop audit sample: pick ten findings and verify that each has an owner, a ticket, a timestamp, and a closure proof. That is stronger evidence than a dashboard with a green percentage.