Model quality, coverage criteria and test selection from models
What model quality means
A usable test model is syntactically well formed in its notation, semantically correct for the intended behavior, complete for the selected scope, internally consistent (no contradictory transitions or unreachable dead ends), unambiguous and traceable to its sources. Verification asks whether the model is built correctly; validation asks whether it is the right model. Validation requires domain experts, because a model can be perfectly formed and still encode the wrong rule.
Coverage and selection criteria
- Structural model coverage: states, transitions, transition pairs, paths, decisions, guards, definition-use associations.
- Data-oriented selection: boundary values and equivalence classes over model variables.
- Requirement-based and risk-based selection: tests tied to selected requirements or high-risk behavior.
- Random and mutation-based selection: mutation introduces small model faults and selects tests that detect them. Coverage is necessary but not sufficient: the abstraction gap means 100% model coverage can still leave code untested.
Worked example
A model has states A, B, C and D with transitions A-B, B-C, C-A, A-D, D-A and B-D. State coverage needs all four states visited; transition coverage needs all six transitions executed. One path, A-B-C-A-B-D-A, executes A-B, B-C, C-A, A-B, B-D and D-A: all six transitions and all four states, in a single test. Transition-pair coverage is not achieved, because pairs such as C-A-B are covered but D-A-B is not. The reasoning: count model elements first, then chain them into feasible paths, then check the next stronger criterion separately.
Traps that cost marks
- Claiming adequacy from 100% transition coverage while ignoring data values and invalid inputs.
- Ignoring infeasible paths and reporting a permanent coverage gap instead of documenting it.
- Measuring coverage without checking which model elements no test reaches; unreached elements may be dead model parts or missing tests.
- Skipping model validation, then blaming the generator for weak tests.