Data Analytics, Data Quality and Decision Support
Core concepts
Analytics maturity runs descriptive (what happened), diagnostic (why), predictive (what is likely) and prescriptive (what should be done). Techniques include profiling, duplicate and gap testing, trend and ratio analysis, benchmarking, regression and classification, and visualisation dashboards.
A candidate must be able to interrogate data quality across recognised dimensions: completeness, accuracy, validity, uniqueness, consistency, timeliness and integrity. Data flows through extraction, transformation and loading (ETL); each step can introduce error. Information produced by the entity (IPE) - any report or extract used as audit or decision evidence - must itself be tested for completeness and accuracy, using record counts, control totals, query/logic review and reconciliation to source.
Computer-assisted audit techniques (CAATs) allow full-population testing rather than sampling. This changes the nature of conclusions: a 100% test can identify every exception, but it also generates false positives where data is dirty or logic is imprecise, and it does not by itself prove that the recorded population is complete.
Traps in questions
- Assuming analytics 'proves' something without validating the underlying data's completeness and accuracy.
- Confusing correlation with causation when interpreting trend dashboards.
- Choosing the wrong quality dimension - e.g. describing an out-of-date balance as 'inaccurate' rather than 'not timely'.
- Forgetting that sampling risk disappears under full-population testing but non-sampling risk (bad logic, poor data) remains.
Worked example
An analyst screens 2 million payments for duplicates and finds 400 matches. Before reporting, they reconcile the extract record count to the ledger transaction count, confirm the date range and currency filters, review the matching logic (same vendor, same amount, within 3 days) and investigate a sample of hits. Only then can they conclude on the population - otherwise the test proves only that the extract is internally consistent.
Rule of thumb: validate the data before you interpret the result; a clean answer from dirty data is still wrong.