DevOps Continuous Testing Strategies
This domain tests your ability to choose a testing strategy across the pipeline: which tests to run, where, how often, and against which risks, including testing in production.
Core rules and concepts
- Risk-based testing: allocate effort by likelihood and impact of failure. High-risk areas get deeper coverage and tighter gates.
- Test early and often: static analysis, unit and component tests close to the code; integration and contract tests at service boundaries; end-to-end and UI tests sparingly; exploratory testing throughout.
- Test pyramid: many fast, isolated unit tests; fewer service and integration tests; very few slow UI tests. The ice-cream-cone anti-pattern inverts this and slows feedback.
- Shift-right: canary releases, blue-green deployment, synthetic monitoring, chaos engineering and production observability to test real conditions.
- Contract testing for microservices reduces dependence on brittle end-to-end suites.
- Service virtualization and test data strategy remove environment bottlenecks.
- Quality gates are risk-based; they block only what they must, at the stage where the risk is cheapest to catch.
Worked example
A team releases a payment service weekly. Unit tests cover calculation rules on every commit; Pact contract tests verify provider-consumer agreements on every pull request; a small smoke suite runs on staging deployment; performance tests run weekly; a 1% canary with synthetic payment checks runs in production. Reasoning: each risk is tested where it is cheapest and fastest to detect. Full end-to-end payment tests are not run on every commit because they are slow and brittle; contract tests plus a canary cover integration and real-traffic risk more efficiently.
Traps that cost marks
- Choosing end-to-end UI tests as the primary regression strategy.
- Ignoring non-functional testing, especially performance and security, until release week.
- Treating shift-right as only monitoring; it also includes controlled experimentation.
- Applying identical gates to all changes instead of risk-based gates.
- Neglecting test data and environments, which makes the strategy undeliverable.
- Forgetting exploratory testing in a strategy that is otherwise automation-heavy.