Fixed-Price Suppliers and a Ten-Week SIT Window
A government benefits programme runs four supplier workstreams - identity, calculation engine, payment and notifications - under fixed-price waterfall contracts, while the in-house integration team works in three-week sprints. System integration testing has a fixed ten-week window before an immovable go-live date set by legislation. Each supplier performs its own factory acceptance test and then delivers into a shared SIT environment that is also refreshed weekly with anonymised production data. Last cycle, three of four suppliers delivered late; SIT lost 17 of its 50 working days waiting for stable interfaces, and 60% of the defects found in SIT traced back to interfaces that had never been tested end to end by anyone. Regression is manual, 900 cases, and takes 14 working days. There is no contractual obligation on suppliers to test interfaces beyond their own boundary, and no environment service-level agreement.
1. Define the test levels, owners and entry/exit criteria across the supplier contracts and the in-house sprints, and explain how you would get them accepted mid-programme.
Read the worked answer
Map the levels to who can actually control them:
- Component and factory acceptance (supplier-owned): entry - build under configuration control, unit coverage met, supplier's own interface contract tests green. Exit - zero open severity 1-2 defects in the delivered scope, published test evidence and a frozen interface version.
- Interface/integration (jointly owned, programme-led): entry - both sides of the interface deployed at the agreed version, consumer-driven contract tests passing, test data loaded, environment confirmed available. Exit - every interface in the interface catalogue exercised end to end at least once at the current version, no open data-integrity defect.
- SIT end-to-end business scenarios (programme-owned): entry - integration exit met, environment SLA in force. Exit - all high-risk scenarios executed, defect thresholds met, residual risk accepted by the business owner.
Getting this accepted mid-programme is a commercial, not a technical, task. Do not ask suppliers to accept new obligations as goodwill: they will refuse. Instead, present the interface contract tests and environment SLA as risk reduction for them - each interface defect found in SIT costs them under the current warranty arrangements and delays their payment milestone. Offer the pragmatic trade: no change to scope or price, but named interface test evidence becomes a condition of the milestone acceptance certificate that already exists in their contract. Route it through the programme board with the legislation date as the compelling event, and get the SIT entry criteria written into the next contract variation rather than relying on working-level agreement.
2. SIT lost 17 of 50 days last cycle and manual regression takes 14 days. Propose how to shift integration testing left and recover schedule, prioritising what you would do first.
Read the worked answer
Attack the two biggest causes in order: unstable interfaces and a regression suite that consumes the window it is meant to protect.
First - stop discovering interface breakage in SIT. Introduce consumer-driven contract testing so each supplier validates its side of the interface against a published contract in its own pipeline, and require the contract test result as part of the delivery evidence. Where a dependent service is unavailable, use service virtualisation with contracts as the source of truth, so end-to-end tests are no longer gated on four suppliers being simultaneously ready. Add a weekly 30-minute interface readiness checkpoint with a named owner per interface: version deployed, contract tests green, data loaded. The reasoning is that 60% of SIT defects came from interfaces nobody tested, so testing them earlier removes both the defects and the queueing.
Second - make regression fit the sprint. Do not automate 900 cases. Select the 150-200 cases covering high-risk functions and interfaces, automate those, and run them continuously per build. Target a two-day regression pack for the SIT window, with the remainder executed risk-based and only where change impact justifies it. Fourteen days of manual regression inside a ten-week window is not a testing plan, it is a schedule hazard.
Third - sequence SIT by risk. Order scenarios so that legally critical and high-risk end-to-end flows run in weeks 1-5, leaving buffer for rework. Do not follow the original document order.
3. Three of four suppliers are late again. What control levers do you have inside the ten-week window, and what do you escalate?
Read the worked answer
Sort levers into those you control and those only the programme board can pull.
Within your control: re-sequence SIT so supplier-independent scenarios (notifications, calculation engine internal flows) run first and the dependent end-to-end chains are placed later; use service virtualisation to test against a stable reference implementation of any missing component; tighten daily defect triage to a 24-hour turnaround for integration defects; and run the automated regression per build so that when a late component arrives it is tested in hours, not days.
Board-level escalation: the immovable legislative date conflicts with contractual delivery dates that have already slipped. The board needs to make one of three choices explicit - reduce scope at go-live (which business scenarios move to a post-go-live release), mandate overtime and additional supplier resource at their cost, or accept a controlled go-live with defined workarounds and a remediated risk register signed by the business owner. Do not present the problem without the options and their consequences.
Evidence to bring: interface readiness status per supplier, the count of high-risk scenarios not yet executed, the projected SIT completion date with a range, and the residual business risk of each option. Also record the schedule impact of each late delivery against the contract for later commercial discussion - not as a threat, but so that the reality of the impact is documented while it happens rather than reconstructed afterwards.