DataX flashcards

33 cards. Read the term and the definition here. The deck at the top of this page is optional.

Data steward

Business-side role accountable for a data domain's definitions, quality rules, and issue resolution. The data owner separately holds budget authority and approves policy.

Master data

Shared, relatively stable entities such as customer, product, supplier, and location that multiple systems reference. Managed centrally to prevent conflicting versions across applications.

Reference data

Controlled code sets used to classify or categorize other data, such as country codes, currency codes, and order statuses. Changes require governed approval and coordinated distribution.

Data lineage

Technical metadata tracing a data element from source systems through transformations to reports. Used for impact analysis, root-cause debugging, and regulatory audit trails.

Third normal form

A relational table is in 3NF when it is in 2NF and has no transitive dependencies: every non-key attribute depends only on the key, not on another non-key attribute.

Star schema

Dimensional model with a central fact table holding measures and foreign keys, surrounded by denormalized dimension tables. Optimized for simple, fast analytical queries.

Fact table grain

The precise meaning of one fact row, such as one order line per day. Declared before adding measures or dimensions to prevent double counting.

SCD Type 2

Slowly changing dimension technique preserving history by inserting a new row per change, with effective and expiry dates plus a current-record flag.

Conformed dimension

A dimension table shared identically across multiple fact tables or data marts, enabling consistent drill-across reporting on the same business entity.

Idempotent pipeline

A pipeline whose rerun leaves the target in the same state as a single run. Achieved with MERGE/upsert on keys or partition overwrite rather than blind appends.

Change data capture

Technique extracting only rows that changed since the last load. Log-based CDC reads database transaction logs with minimal source impact; timestamp-based CDC relies on an updated column.

Watermark

Stored high point, such as maximum timestamp or sequence, marking what an incremental load has processed. Late records older than it need a lookback window.

ELT versus ETL

ETL transforms before loading into the warehouse; ELT loads raw data first and transforms using target compute. ELT suits scalable cloud columnar warehouses.

Late-arriving data

Records that reach the pipeline after their expected period closed. Handled by lookback reprocessing, partition rewrites, or accepting them into the current period.

Data completeness

Quality dimension measuring whether required values are present. Computed as non-null, non-blank records divided by total records, reported per critical column.

Fuzzy matching

Record-linkage approach scoring string similarity, for example Levenshtein distance or Jaro-Winkler. Pairs above a threshold are candidates for deduplication or merge.

IQR outlier rule

Flags values below Q1 minus 1.5 times the interquartile range or above Q3 plus 1.5 times it. Distribution-free and robust to extreme values.

MNAR

Missing not at random: the probability of a missing value depends on the unobserved value itself. Naive imputation then biases results, so sensitivity analysis is needed.

Referential integrity check

Counts child rows whose foreign key lacks a matching parent key. Orphan counts reveal load-order problems or source-side deletions.

Type I error

Rejecting a null hypothesis that is actually true, with probability alpha. Lowering alpha reduces false positives but raises Type II error risk.

Statistical power

Probability of correctly rejecting a false null hypothesis, equal to 1 minus beta. Conventionally targeted at 0.80 and increased by larger samples or bigger effects.

Standard error of mean

Standard deviation of the sampling distribution of the mean, equal to sigma divided by the square root of n. Shrinks as sample size grows.

Simpson's paradox

A trend present within every subgroup reverses or disappears when groups are pooled. Caused by an unbalanced confounding variable across groups.

Bonferroni correction

Controls family-wise error by testing each of k hypotheses at alpha divided by k. Conservative, reducing power as the number of tests grows.

Truncated axis

A value axis that does not start at zero. Acceptable for line charts of change, misleading for bars because length no longer encodes magnitude proportionally.

Sequential color palette

Color scheme ordered from light to dark representing increasing magnitude of a single variable. Diverging palettes instead highlight deviation around a meaningful midpoint.

Data-ink ratio

Tufte's principle that most ink in a graphic should represent data. Redundant gridlines, 3D effects, and decoration lower the ratio and obscure patterns.

Small multiples

Repeating the same chart type and scale across panels, one per category. Avoids dual axes and lets viewers compare patterns without visual clutter.

Tokenization

Replacing sensitive values with non-reversible tokens stored in a protected vault. Preserves referential links for processing while removing the original value from downstream systems.

k-anonymity

Privacy guarantee that each record is indistinguishable from at least k minus one others on quasi-identifiers. Weak against attackers with additional background information.

ABAC

Attribute-based access control granting access from policies evaluated against user, resource, action, and environment attributes. More granular than role-based control.

Columnar storage

Stores values of each column contiguously, enabling high compression and reading only referenced columns. Favors analytical scans over single-row transactional access.

Partition pruning

Query optimizer skipping partitions whose key values cannot satisfy the filter, such as a date range. Requires filters on the partition key to take effect.