DEA Level 3 cheat sheet

A data governance operating model defines decision rights (who decides), accountability (who is answerable), and controls (how compliance is verified) for each data domain.

Data steward: accountable for data quality and definitions within a domain; data owner: accountable for access, classification, and risk decisions; data custodian: operates the technical controls.

A data quality dimension set typically covers completeness, accuracy, consistency, timeliness, uniqueness, and validity; each is measured as a ratio of conforming records to total records.

Data quality score = (records passing all defined rules / total records assessed) x 100; report per dimension, not as a single blended number, to preserve diagnosability.

A business glossary defines business terms and their approved definitions; a data dictionary defines physical structures (tables, columns, types); a data catalog indexes both plus lineage and ownership.

A critical data element (CDE) is a field that materially affects reporting, regulatory submission, or a key business process; CDEs receive mandatory stewardship, quality thresholds, and lineage documentation.

Issue management follows a lifecycle: log, triage/classify, assign root-cause owner, remediate, verify, close; root-cause categories are commonly process, system, data entry, and design.

Data governance metrics are lagging (issues resolved, audit findings) or leading (CDEs with assigned stewards, glossary coverage, rule coverage); mature programs track both.

A data domain is a subject-area partition of data (customer, product, finance) that carries its own owner, definitions, quality rules, and access policy.

Normalization: 1NF removes repeating groups; 2NF removes partial dependencies on part of a composite key; 3NF removes transitive dependencies on non-key attributes; BCNF requires every determinant to be a candidate key.

Dimensional model: a fact table holds measures plus foreign keys at a declared grain; dimension tables hold descriptive attributes; the grain statement must be fixed before choosing measures.

Star schema keeps dimensions denormalized (fewer joins, simpler queries); snowflake schema normalizes dimensions (less redundancy, more joins).

Slowly changing dimension Type 1 overwrites (no history), Type 2 adds a new row with version/effective dates and a current flag, Type 3 adds a prior-value column (limited history).

Kimball (dimensional, bottom-up, bus matrix of conformed dimensions) and Inmon (normalized enterprise warehouse, then departmental marts) differ in build order and core structure, not in end goal.

Data Vault models use hubs (business keys), links (relationships), and satellites (descriptive, time-stamped attributes); designed for auditability and parallel loading.

CAP theorem: under a network partition a distributed system must choose consistency or availability; PACELC extends this to latency versus consistency trade-offs when no partition exists.

Lambda architecture runs batch and speed layers in parallel with a serving layer; Kappa architecture drops the batch layer and reprocesses the stream for all views.

Conceptual, logical, and physical models differ in abstraction: conceptual = entities and relationships, logical = attributes and keys independent of technology, physical = tables, indexes, partitions, and storage.

ETL transforms before loading into the target; ELT loads raw data first and transforms inside the target platform, exploiting its compute and preserving raw history.

Idempotency rule: a pipeline rerun must produce the same target state; implement via MERGE/upsert on a business key, partition overwrite, or deduplication on a run identifier.

Change data capture (CDC) options: log-based (reads the database transaction log, low impact, near-real-time), trigger-based (write overhead), and snapshot-diff (full compare, expensive at scale).

Exactly-once delivery is achieved in practice by at-least-once delivery plus idempotent writes or deduplication keys; true end-to-end exactly-once requires transactional sinks.

Watermarks in stream processing define how long to wait for out-of-order events before emitting a window result; allowed lateness controls how late events are still accepted.

Batch versus streaming decision rule: choose streaming when the business action depends on data freshness measured in seconds; otherwise batch is cheaper and simpler to operate.

Orchestration dependencies should be data-driven (sensor on partition arrival) rather than time-driven where upstream latency varies; retries need backoff and a dead-letter path.

Data pipeline observability covers freshness, volume, schema, and distribution checks; a freshness SLA is measured from source event time or source commit time to target availability.

Partitioning and file sizing: target files in the 128 MB-1 GB range for columnar formats to avoid the small-file problem; partition on low-cardinality, frequently filtered columns.

Confusion matrix metrics: precision = TP/(TP+FP); recall = TP/(TP+FN); F1 = 2PR/(P+R); accuracy is misleading under class imbalance.

ROC-AUC measures ranking quality across all thresholds and is insensitive to class balance; PR-AUC is preferred when the positive class is rare.

Bias-variance: high bias underfits (train and validation error both high); high variance overfits (low train error, high validation error); remedies are more features/complexity versus regularization/more data.

Regularization: L1 (lasso) drives coefficients to zero and performs feature selection; L2 (ridge) shrinks coefficients smoothly; elastic net mixes both with ratio parameter.

Cross-validation: k-fold for general data, stratified k-fold for imbalanced classes, time-series split for temporal data to prevent leakage from future to past.

Data leakage occurs when training features encode target information unavailable at prediction time (e.g., post-outcome fields, target-encoded variables computed on full data).

Hypothesis testing: p-value is the probability of observed-or-more-extreme data under the null; it is not the probability the null is true. Alpha is set before the test.

Model monitoring tracks input drift (population stability index, distribution tests) and output drift (score distribution, label-based performance once outcomes arrive).

Regression diagnostics: residual plots for linearity and homoscedasticity, variance inflation factor above roughly 5-10 flags multicollinearity, Durbin-Watson for autocorrelation.

Data classification tiers (public, internal, confidential, restricted) drive control requirements: encryption, access approval level, retention, and sharing restrictions.

Encryption states: at rest (storage-level or column-level), in transit (TLS), and in use (tokenization, secure enclaves); key management is the control that makes encryption meaningful.

Access control models: RBAC grants by role, ABAC by attributes and context, and purpose-based access ties entitlement to a declared processing purpose.

Privacy principles common to GDPR-style regimes: lawfulness, purpose limitation, data minimization, accuracy, storage limitation, integrity/confidentiality, accountability.

Data subject rights typically include access, rectification, erasure, restriction, portability, and objection; each requires an identified system of record and a response SLA.

Pseudonymization replaces direct identifiers so re-identification needs extra data held separately; anonymization must make re-identification not reasonably possible, and is a higher bar.

Masking options: static (irreversible, applied in non-production copies), dynamic (applied at query time), and format-preserving tokenization that keeps referential joins possible.

Least privilege and separation of duties: production data access is granted per task, time-bound where possible, and privileged access is logged and reviewed on a schedule.

Breach notification timelines vary by jurisdiction; the operational control is a documented incident runbook with detection, containment, assessment, notification, and post-incident review.

A data strategy links business objectives to data capabilities via explicit use cases; each use case names the data, capability, owner, and measurable outcome.

Data maturity models progress from ad hoc to defined to managed to optimized; assessment scores per capability, not a single enterprise score, drive the roadmap.

Value case for a data initiative should state benefit type (revenue, cost, risk), baseline, target, and measurement method; unmeasured benefits are not fundable.

Prioritization rule: rank by value divided by effort with dependencies and risk as gates; a use case blocked by a missing CDE is sequenced behind the CDE remediation.

Operating model choices: centralized (single team), federated (domain teams with a central standard-setter), and hybrid; the hybrid is the common target for scaled organizations.

Delivery approaches: waterfall for fixed-scope regulatory work, agile/iterative for discovery-heavy analytics, and data product teams with a published contract and SLO for platform work.

A data product has an owner, a defined output, a quality SLO, documented schema/contract, and consumers; without an SLO it is a dataset, not a product.

Change management adoption levers: sponsor sponsorship, role-based training, incentives aligned to new process, and feedback loops; adoption is measured by usage, not attendance.

Program risk register entries need likelihood, impact, owner, mitigation, and trigger; data programs most often fail on ownership gaps and unclear decision rights, not tooling.