CDGP cheat sheet

Data governance is the exercise of authority and control (planning, monitoring, enforcement) over data management to ensure data is managed as an asset.

Governance defines decision rights and accountability; data management executes the decisions. Governance answers what and who, management answers how.

Policy hierarchy: policy (mandatory rule) → standard (measurable requirement) → procedure (step-by-step) → guideline (recommendation); lower levels must conform to higher.

Role decision rule: data owner is accountable for a domain; data steward is responsible for day-to-day quality and metadata; data custodian implements technical storage and security.

Typical governance bodies: Data Governance Council (strategic decisions), Data Stewardship Committee (operational coordination), Data Quality Council (issue prioritization).

Decision rights are documented in a RACI matrix mapping activities to Responsible, Accountable, Consulted, and Informed parties.

Governance KPIs include policy compliance rate, stewardship coverage (percent of domains with assigned stewards), and mean time to resolve data issues.

Governance maturity progresses from initial/ad hoc → managed → defined → measured → optimized; each level requires documented policies, metrics, and enforcement.

Data management is the development, execution, and supervision of plans, policies, programs, and practices that deliver, control, protect, and enhance data assets across their lifecycle.

DAMA-DMBOK2 defines eleven knowledge areas: Data Governance, Data Architecture, Data Modeling and Design, Data Storage and Operations, Data Security, Data Integration and Interoperability, Document and Content Management, Reference and Master Data, Data Warehousing and BI, Metadata, Data Quality.

Lifecycle stages: plan → design → enable → create/obtain → maintain/use → enhance → dispose/archive; controls and metadata apply at every stage.

ACID transaction guarantees: Atomicity (all-or-nothing), Consistency (valid state transitions), Isolation (concurrent transactions appear serial), Durability (committed changes survive failure).

CAP theorem: a distributed system can simultaneously guarantee only two of Consistency, Availability, and Partition tolerance; under a network partition, choose consistency or availability.

RPO = maximum tolerable data loss measured in time; RTO = maximum tolerable time to restore service. RPO drives backup frequency, RTO drives recovery architecture.

Data categories: structured (rows/columns), semi-structured (JSON, XML with tags), unstructured (text, images, audio); management approach differs by structure.

Master data describes core business entities (customer, product); reference data is a controlled set of allowed values (country codes); transactional data records events.

Integration patterns: ETL (transform before load), ELT (transform after load), CDC (capture changes), and streaming (continuous event processing).

DAMA's six core dimensions: accuracy (reflects reality), completeness (required values present), timeliness (available when needed), validity (conforms to rules), consistency (same across sources), uniqueness (no duplicates).

Completeness percent = (non-null required values ÷ total required values) × 100.

Validity percent = (records passing a rule ÷ total records evaluated) × 100; rules test format, range, code-list membership, or cross-field logic.

Uniqueness is measured as duplicate rate = (duplicate records ÷ total records); a natural or business key defines the duplicate test.

Consistency checks compare the same attribute across systems or against derived values; inconsistency is a mismatch beyond a defined tolerance.

Data profiling statistics: null count, distinct count, min/max, frequency distribution, pattern discovery, and key uniqueness; profiling precedes rule definition.

DQ issue lifecycle: detect → log → analyze root cause → remediate (correct at source preferred) → verify → monitor for recurrence.

Root causes cluster into data entry, integration/transformation, migration/conversion, and process/definition gaps; correcting at source prevents recurrence.

A data quality scorecard aggregates dimension scores with weights; a record is good only if it passes all mandatory rules.

Metadata is data about data: it describes content, context, structure, quality, and lineage of data assets to enable discovery, understanding, and control.

Three metadata types: business (definitions, glossary, ownership), technical (schemas, tables, ETL logic), operational (job logs, run times, access counts).

Metadata lifecycle: create/capture → maintain → integrate → use → archive; metadata must be versioned and owned like other data.

A metadata repository stores technical and operational metadata; a data catalog provides search, business glossary, lineage, and collaboration on top.

Lineage types: data lineage (record/field movement), transformation lineage (logic applied), and business lineage (report-to-source traceability).

ISO/IEC 11179 governs metadata registries and data element naming; Dublin Core covers document metadata; both provide standardized attribute definitions.

Metadata quality attributes include accuracy, completeness, currency, and consistency; stale metadata undermines catalog trust.

Core metadata uses: impact analysis before change, data discovery for analysts, regulatory audit trails, and duplicate data identification.

A business glossary term should have a definition, owner, synonyms, related terms, and associated data assets.

Data architecture components: data models, data flows, data lifecycle, storage, and integration patterns; it defines how data is structured, stored, integrated, and used.

Modeling levels: conceptual (business entities/relationships), logical (attributes, keys, normalized, technology-independent), physical (tables, types, indexes, partitions).

ER constructs: entity, attribute, relationship, cardinality (1:1, 1:M, M:N), and optionality; keys are unique identifiers (natural or surrogate).

Normalization: 1NF removes repeating groups; 2NF removes partial dependencies on a composite key; 3NF removes transitive dependencies on non-key attributes.

Dimensional model: fact table holds measures and foreign keys at a declared grain; dimension tables hold descriptive context; star = denormalized dimensions, snowflake = normalized dimensions.

SCD types: Type 1 overwrites (no history), Type 2 adds a new row with version/effective dates (full history), Type 3 adds a previous-value column (limited history).

Data vault: hubs (business keys), links (relationships between hubs), satellites (descriptive attributes and history); designed for auditability and parallel loading.

Architecture frameworks: Zachman (ontology of artifacts), TOGAF (ADM method), DAMA (data management context); each organizes architecture artifacts differently.

CRUD matrix maps business processes to data entities and records Create/Read/Update/Delete access, revealing ownership and integration requirements.