The Corporate Test Policy Waiver at FinSecure
FinSecure Group publishes a corporate test policy requiring risk-based testing, independent system testing performed by a central test factory, and one defined defect severity scale. The new mobile payments programme - four Scrum teams, two-week sprints, a quarterly release train - was given a waiver to keep its testers inside the delivery teams. Six months in, the central test factory reports that mobile releases arrive with 40% of their system-test entry criteria unmet, the production escape rate has risen from 6% to 17%, and the two test organisations exchange defect reports using different severity scales, so triage arguments now consume about a third of each joint meeting. The CIO wants a single 'test approach' for the programme; the central factory manager replies that the policy already defines it and that the waiver is simply non-compliance. You are the newly appointed programme test manager, accountable to both.
1. Explain what belongs in the organizational test strategy and what belongs in the programme test approach in this situation, and name the three conflicts you must resolve first.
Read the worked answer
The organizational test strategy is the stable, enterprise-wide asset: the test policy, the required test levels and the degree of independence expected, the corporate risk classification scheme, the severity scale, the metrics taxonomy, and the audit and reporting obligations. It applies to every programme and should not be rewritten for one release train.
The programme test approach is the deliberate instantiation of that strategy for this context: which test levels the four Scrum teams own, what 'independent' concretely means when testers sit in the team, how the corporate severity scale maps to the backlog priority field, and which entry criteria are meaningful for a sprint increment.
Three conflicts to resolve first:
- Independence. Embedded testers are not organisationally independent. Define compensating controls instead of pretending otherwise - cross-team review of test design, a central risk-based exploratory and regression service, and a test manager reporting outside the delivery line.
- Entry criteria. The factory's document-based criteria cannot be met by a sprint increment. Replace them with a definition of ready plus automated quality gates, and state the equivalence in writing.
- Severity. One scale, one documented mapping, one triage forum, so defect data becomes comparable across both organisations.
The reasoning is that the strategy states what must be achieved and the approach states how this context achieves it. Deviation is legitimate only where the approach supplies equivalent or stronger evidence - and that case has to be written down and reviewed, not assumed.
2. Design the governance that stops the waiver becoming permanent non-compliance, and specify the metrics you would use to show embedded testing is at least as rigorous as the central factory.
Read the worked answer
Convert the waiver into a time-boxed, evidence-backed exception rather than an open-ended permission. Concretely: a one-page derivation record listing which strategy elements apply unchanged, which are tailored, and what compensating control replaces each; an exception formally owned by the CIO with a two-quarter expiry and review at each release-train boundary; a joint forum (programme test manager, factory manager, quality function) meeting monthly with authority to revoke; and a single severity scale with one triage route to end the mapping arguments.
Evidence that embedded testing is no less rigorous must be metric-based, not testimonial:
- Risk coverage: share of high-risk items with tests at the required level, measured on the same risk register the factory uses.
- Phase containment: defect detection percentage per level, and the share of defects found before system integration.
- Escapes: production escape rate and the severity mix of escapes, baselined now at 17% against the 6% corporate norm.
- Gate compliance: entry and exit criteria met against the tailored definitions, not the factory's.
- Defect data quality: proportion of defects with agreed severity and confirmed root cause area.
Set decision rules in advance - for example, escapes no worse than 6% across two release trains, and no high-risk item without end-to-end coverage. If two consecutive reviews miss the thresholds, the exception lapses for that service. That converts an argument about philosophy into a controlled experiment with an exit condition.
3. The four Scrum teams object that any central regression or cross-team review will slow them down. How do you make the tailoring stick with them rather than impose it?
Read the worked answer
Treat adoption as a change problem, not a compliance problem. First, make the cost visible and bounded: agree a fixed regression service slot per sprint (for example, a 24-hour automated regression run on the release candidate) so teams can plan around it, rather than an open-ended review queue. Second, give the teams something they want in exchange - defect data in their own tooling, fast feedback on interface breakages, and the right to challenge the corporate severity scale via the joint forum.
Third, make the accountabilities explicit rather than cultural. Name a test lead per team who owns risk coverage for that service, and a central counterpart who owns end-to-end and regression coverage. Publish a small RACI so that 'who signs the exit criteria' has one answer.
Fourth, run a pilot on the highest-risk service for one release train and let the measured outcome (escapes, phase containment, cycle time of regression feedback) drive the rollout decision. Teams accept evidence about their own product far more readily than a policy citation.
Finally, escalate only on decision rights, not on style. If a team refuses to meet a gate that the exception record says is mandatory, that is a governance failure to be raised at the forum - not a debate about methodology. Keeping those two categories separate is what makes the tailoring durable.