Data Quality, Integrity, and Validation
The dimensions
Classic data quality dimensions: completeness, accuracy, timeliness, consistency, validity, and uniqueness. In healthcare, add plausibility and conformance to clinical rules. Integrity means the data is unaltered and traceable (audit trails, referential integrity, no orphan records). Validation is the set of checks applied at entry and downstream.
Concepts to master
- Types of checks: range/domain checks, format checks, cross-field (consistency) checks, referential integrity, duplicate detection, and reconciliation against an external source of truth.
- Edit checks at point of entry vs. batch validation after the fact; the tradeoff between blocking errors and workflow burden.
- Data profiling to discover issues before designing rules.
- Root-cause analysis: distinguish a data entry error, a system/interface error, and a definitional mismatch.
- Data quality dashboards, thresholds, and exception queues; stewardship of flagged records.
- Provenance and audit trails as integrity controls.
Traps
Questions love to conflate validity (conforms to a rule/format) with accuracy (reflects reality). A value can be perfectly valid and completely wrong. Another trap: assuming more validation is always better-overly strict edits cause workarounds and silent data loss. Also watch for questions that describe a mismatch between two systems and expect "data entry error" when the real cause is a mapping or definition problem.
Rule of thumb
Before writing a validation rule, profile the data and ask what failure mode the rule prevents. If the same value fails in two systems but is correct in the source, the problem is integration or definition, not data entry.
Worked example
A birth date of 1850 for a living adult passes a format check but fails a plausibility/range check. Catching it requires a rule tied to clinical context, not just a date mask-illustrating why context-aware validation beats format-only validation.