Data Quality & Observability
Data Quality & Observability - what the exam tests
This domain tests your ability to pick the right quality operation for a symptom, read a scorecard correctly, and keep pipelines observable after go-live. Questions mix Cloud Data Quality assets (rules, scorecards, cleansing and matching transformations) with Data Observability signals such as freshness, volume, schema drift, and anomalous values.
Core rules
- Quality dimensions: completeness, validity, accuracy, consistency, uniqueness, timeliness. Each rule maps to one or more dimensions; a scorecard aggregates rule results into a score.
- Cleansing transformations: Standardizer normalizes values; Address Validator verifies country-specific addresses; Labeler identifies data types such as name or phone; Parser splits free text; Key Generator builds match keys; Match groups records; Consolidation picks a survivor.
- Match produces match groups with scores; thresholds separate automatic matches from review. Consolidation is a separate step that creates the golden record.
- Exception handling routes rejected rows to an exception table or a human task - they are never silently dropped.
- Observability monitors the pipeline as well as the data: job-duration anomalies, row-count deviation, null-rate spikes, schema changes. Alerts fire on static thresholds or learned baselines.
Worked example
A daily load shows a 40% row-count drop and a null spike in postal_code. Reasoning: the null spike is a validity/completeness breach in the data, while the row-count drop is a volume anomaly in the pipeline. Check the source extract first for upstream truncation, then inspect the mapping filter. A volume alert would have fired before the scorecard did. Fix the extract and re-run; do not loosen the rule threshold to make the score green.
Traps
- Confusing Match with Consolidation: Match groups, Consolidation survives.
- Reporting a single rule result as the whole dimension score.
- Treating an exception row as a failure - it is a routed record awaiting stewardship.