CCSP study materials
CCSP study resources

Making the Pipeline Gate Credible

CCSP · Put your knowledge to work in context.

All case studies

Making the Pipeline Gate Credible

Tessera Retail ships 60 times a day across 140 microservices on a managed Kubernetes platform. Eight months ago AppSec enabled SAST, dependency scanning, and container scanning; the backlog now holds 1,400 critical findings, 92% of them in three legacy services nobody has touched in a year. Developers hold a standing exception to skip the gate because it 'blocks releases'. A recent incident involved a serverless order-webhook function invoked from an unauthenticated API route, whose execution role allows s3:* and whose third-party secret sits in a plaintext environment variable. Leadership wants the finding count down 80% this quarter.

1. Leadership wants the 1,400 findings reduced by 80%. How do you handle that request?

Read the worked answer

Refuse the metric as stated and propose a different one.

An 80% count reduction is achievable in a week by deleting the three legacy services or suppressing their findings. Neither reduces risk, and both destroy the program's credibility - which is the actual problem. Separate the two things leadership is asking for.

Triage model, applied in order:

  1. Reachability and data path. Is the finding in code that is internet-facing or handles regulated data? A critical CVE in a build-time-only library on an unreachable service is noise. This alone should reclassify a large share of the 1,400.
  2. Deduplicate by root cause. 1,400 findings usually means a few dozen causes: one base image, one transitive dependency, one repeated pattern. Fix by root cause; the count falls as a side effect and the fixes are real.
  3. Decide disposition per legacy service. Decommission, or accept with a named owner and a date. Idle legacy code is a deletion decision, not a remediation decision - and deleting the service is a legitimate route to 80%, unlike suppressing the finding.
  4. Debt budget. A fixed remediation allocation per sprint; unowned findings default to the service owner's queue.

Proposed KPI: percentage of critical findings in production, reachable data paths remediated within SLA, plus mean time to remediate. Report the raw count for transparency and the SLA metric for judgement.

2. Redesign the order-webhook function. Identify each distinct fault and fix it at the layer it belongs to.

Read the worked answer

Five separate faults; fix each where it lives.

  1. Unauthenticated invocation is an application-layer problem, not a network-placement problem. Verify the sender: HMAC signature over the raw body using a shared secret, a timestamp with a replay window, and rejection of unsigned requests.
  2. Over-privileged role. Replace s3:* with PutObject on one bucket prefix plus the logging permissions actually used. Derive from the call graph; do not copy the old policy.
  3. Secret in a plaintext environment variable is doubly wrong in serverless: environment variables are readable by anyone who can read function configuration, and they surface in logs, traces, and crash dumps. Move to the managed secrets store, fetch at cold start, cache in memory, enable rotation, and grant the execution role read on exactly that secret ARN.
  4. No input validation. Validate schema, size, and origin before touching downstream services - this route is an unauthenticated entry point into the data plane.
  5. No rate limiting. Place it behind the API gateway with quotas so a flood neither scales your bill nor hammers downstream.

Prevent recurrence: the pattern is 'new function, copied role, secret in env'. Add a deployment policy gate that fails on wildcard actions in execution roles and on literal secrets in function configuration, and publish a blessed function template so the easy path is the safe one.

3. Developers bypass the gate under a standing exception. How do you make the gate credible rather than punitive?

Read the worked answer

The bypass exists because the gate produces findings developers cannot act on at the moment they are blocked.

  • Gate where feedback is actionable. Pre-commit secret scanning, dependency scanning on changed manifests only, container base-image policy, IaC policy checks at build: fast, deterministic, mostly true positives. Full SAST runs asynchronously and files tickets; it should not block a deploy.
  • Diff-based gating. Fail on new critical findings introduced by the change, not on inherited debt. A developer who did not create a finding should not be blocked by it; one who did should be.
  • Measure and publish the false positive rate. If a large share of criticals are noise, developers are right to bypass and the program has already lost the argument. Tune the rules and report the number.
  • Replace the standing exception with a register. Self-service, time-boxed, named owner, automatic expiry, weekly report to the CISO. A bypass becomes a recorded risk acceptance with a date on it - a different governance object entirely.
  • Measure escapes. The number that matters is incidents caused by issues the pipeline should have caught, not findings raised.