The Pull-Request Gate Nobody Trusts
Contoso has 300 Git repositories in Azure DevOps. A Jenkins pipeline runs runCxConsole Scan synchronously on every pull request against a per-branch CxSAST project, then fails the build if any new High result appears. Full scans take 25 minutes; the PR pipeline SLA is 15 minutes. To keep runtime down, the team added an exclude for /generated/ and switched the preset to a small High-only preset. After 40 builds failed on results the developers judged false positives (mostly generated code the exclude did not match because of path casing), the release manager disabled the Checkmarx stage. Meanwhile, 300 per-branch projects have accumulated, and the nightly full scan of main now reports 11,000 results. The engineering director wants the gate back on in two weeks without slowing delivery.
1. Diagnose why the gate was disabled and which design choices caused it.
Read the worked answer
The gate failed for three compounding reasons. First, the pipeline scanned synchronously for 25 minutes against a 15-minute SLA, so the team cut coverage to compensate; a gate that cannot finish on time will always be disabled under delivery pressure. Second, per-branch projects have no baseline: the first scan of a new branch project marks every pre-existing High as new, so the gate fired on legacy debt, not on the pull request's changes. Third, the exclude was written as /generated/ but the Windows scan paths use backslashes and different casing, so generated code remained in scope; developers saw the same unactionable results repeatedly.
The High-only preset then hid Medium issues without fixing the baseline problem. Once 40 builds failed for reasons developers could not act on, the stage lost credibility and was removed. None of this proves the gate idea is wrong; it proves the baseline, scope, and feedback loop were wrong. The 300 accumulated projects also show there was no lifecycle policy for scan projects.
2. Redesign the pipeline: scan type, project strategy, gating, and developer feedback.
Read the worked answer
Redesign around a stable baseline and a fast path. Keep one project per repository for main and long-lived release branches. Use the plugin's pull-request scan mode if available, or scan the merge commit into that project and compare new results against the target branch's last scan. Avoid ephemeral per-branch projects except with a TTL and automatic cleanup.
In Jenkins, run the scan in parallel with tests or asynchronously with a status check, using incremental scans where supported and ForceScan only to override the no-changes skip when needed. Gate on new Critical and High results in the PR, with a documented allowlist that expires; report Medium as a comment, not a failure. Fix the exclude with the correct path separator and casing, and verify by pushing a test commit to a generated file before trusting it.
Add the IDE plugin so developers see results before the PR, and tune the PR preset to queries relevant to changed code. Measure p95 PR duration and false-failure rate weekly. Re-enable on one pilot repository first, then expand.
3. How would you migrate the 300 per-branch projects and deal with the 11,000 results on main?
Read the worked answer
Inventory before deleting. Use the CxSAST REST API or a portal export to list projects with last-scan date, owner, and branch; map names to repositories and identify the default branch. Archive or delete projects with no scan in 90 days and no matching branch, after exporting their results for audit. Consolidate to one project per repository plus long-lived release branches, and establish a baseline scan of main before enabling any gate.
Then triage the 11,000 results by grouping on query and sink. Many will collapse to a handful of root causes in generated code, vendored libraries, and one or two helpers. Use query tuning and exclude-folder settings for proven noise, not blanket Not Exploitable. Freeze new project creation behind a naming convention and a TTL.
Finally, re-enable the gate on a pilot repository, compare new-result counts for two weeks, and only then roll out. Report the migration as reduced project count and reduced untriaged results, not as a mass closure. If the baseline scan of main is not clean, the gate will inherit the same credibility problem.