Data Quality and Integrity
Data Quality and Integrity tests whether you can measure data fitness for purpose, trace a defect to its root cause, and prove that data used for reporting and decisions is complete, accurate and unaltered. Items often present a wrong number in a regulatory report and ask what to fix first.
Quality dimensions
Standard dimensions: accuracy (matches reality), completeness (no missing required values), consistency (agrees across systems), timeliness (available when needed), uniqueness (no unintended duplicates), validity (conforms to defined formats and rules). Add conformity to reference data where relevant. Precision is not accuracy - a value can be precise to six decimals and still wrong.
Operationalise quality with profiling to establish baselines, explicit data quality rules with thresholds and alerting, a scorecard by domain, and a named owner accountable for each critical data element. Root-cause analysis separates process defects (a broken join, a missing validation) from source defects (the customer gave a wrong address) - only the first is fixed by engineering.
Integrity as a control property
In a controls sense, integrity means data is complete, accurate, authorised and unaltered in transit and at rest. Enforce it with input validation and edit checks, reconciliation and batch totals between systems, hash totals and checksums, four-eyes approval for manual adjustments, change control on transformation logic, and immutable audit trails. Data lineage - the documented path from source to report - is what makes a regulatory number defensible.
Master data management addresses the duplication and drift that cause most cross-system inconsistency, by defining golden records and stewardship for core entities such as customer, product and supplier.
Worked example
A monthly regulatory return undercounts accounts by three per cent. Reasoning: profile the report and the source to confirm the gap, then trace lineage through each transformation. The defect is an inner join on country code that silently drops records with null country. Actions: correct the join logic under change control so it is reviewed, backfill the affected periods and notify the report owner, add a data quality rule with a row-count reconciliation threshold, and assign the critical data element to an owner. Correcting only the records would leave the pipeline ready to fail again.
Traps that cost marks
Fixing individual records without fixing the pipeline. Measuring quality with no threshold, alert or owner, so nobody acts. Confusing accuracy with precision. Having no lineage for numbers reported externally. Allowing manual adjustments to a report with no second approver and no audit trail.