ACA Big Data key terms

Data whose volume, velocity, or variety exceeds what conventional relational tools can handle affordably, requiring distributed storage and parallel processing architectures to extract value.

A storage layer that spreads files across many machines while presenting a single logical namespace, providing fault tolerance through replication and enabling parallel reads by compute nodes.

A batch programming model that splits work into a map phase producing key-value pairs and a reduce phase aggregating them, allowing computation to run in parallel across a cluster.

A dependency graph of tasks with no cycles, used by schedulers to determine execution order so downstream nodes run only after their upstream dependencies succeed.

A physical layout that stores values of the same column together rather than row by row, improving compression and letting analytical queries read only the columns they need.

Dividing a large table into segments by a column value such as date, so queries that filter on that column scan fewer segments and lifecycle rules can expire old data cheaply.

A retention policy that automatically deletes or archives table partitions after a configured age, controlling storage cost without manual cleanup jobs.

Storing frequently accessed data in fast, higher-cost storage and rarely accessed data in slower, cheaper storage, balancing query latency against total storage spend.

Alibaba Cloud's fully managed data warehouse service that provides massive-scale SQL and batch computation over structured data, with built-in storage, security, and job scheduling.

The core data object in MaxCompute, defined by a schema of typed columns plus optional partitions; it holds data physically and is the target of SQL and tunnel operations.

A MaxCompute job type that runs declarative SQL statements to transform and query warehouse tables, executed by the distributed engine rather than on a single server.

A high-throughput upload and download service used to move bulk data between external systems and MaxCompute tables, bypassing SQL for efficient batch ingestion and extraction.

Custom code registered with the compute engine so SQL statements can call it like a built-in function, extending processing to logic the standard function library does not cover.

Alibaba Cloud's unified platform for data integration, development, scheduling, governance, and operations, tying together compute engines and storage into managed pipelines.

A scheduled collection of dependent nodes in DataWorks that runs on a recurring basis, with upstream and downstream relationships controlling execution order and reruns.

The configured recurrence for a workflow, such as hourly or daily, which determines when instances are generated and which time window of data each run processes.

Each scheduled execution of a node is an instance; rerunning re-executes a specific instance, optionally with its downstream nodes, to repair failed or corrected data.

A DataWorks component that copies data between heterogeneous sources and sinks using configurable readers and writers, supporting both batch synchronization and incremental modes.

Two ingestion strategies: full sync copies an entire source dataset each run, while incremental sync transfers only rows changed since the last successful run using a watermark column.

A monotonically increasing field such as an update timestamp used to identify new or changed records, letting incremental jobs resume from the last processed value.

A technique that reads a database's write-ahead or binlog stream to emit inserts, updates, and deletes as they happen, feeding downstream systems with low latency.

An automated check attached to a table or pipeline, such as null-rate or uniqueness thresholds, that flags or blocks downstream use when results fall outside expectations.

A recorded map of how data flows between tables and jobs, showing which upstream sources produced a given output so impact analysis and root-cause tracing are possible.

The practice of cataloging tables, columns, owners, and schemas so users can discover assets, understand their meaning, and govern access consistently.

Permission rules that restrict access at the row or column level rather than only at the table level, so different users see different slices of the same dataset.

Transforming sensitive values, such as partially hiding a phone number, when results are returned to users lacking full clearance, preserving analytical utility while protecting privacy.

A classification scheme labeling data by sensitivity, combined with user clearance levels, so the platform can automatically allow or deny access to labeled objects.

Continuous processing of unbounded event flows with low latency, where results update as new records arrive instead of waiting for a scheduled batch to finish.

Event time is when an event actually occurred, recorded in the data; processing time is when the system handles it. The gap drives late-arriving data handling.

A progress marker in a stream that tells the engine no events older than a given event-time bound are expected, allowing windows to close despite out-of-order arrivals.

A bounded slice of a stream, such as tumbling, sliding, or session windows, over which aggregations are computed so infinite input yields finite, meaningful results.

A delivery guarantee ensuring each record affects the final result precisely one time, typically achieved through checkpointing and idempotent or transactional sinks.

Periodically persisting the state of a streaming job so it can recover from failure and resume from the last snapshot instead of reprocessing the entire input.

A durable buffer that decouples producers from consumers, holding event streams so downstream processors can read at their own pace and replay past data.

Alibaba Cloud's scalable object store for unstructured data such as files and logs, commonly used as a data lake foundation and as a source or sink for analytics engines.

A central repository holding raw data in native formats at scale, queried on demand by multiple engines, in contrast to a warehouse's pre-modeled structured tables.

A curated analytical store with modeled, schema-enforced tables optimized for SQL reporting, typically fed by pipelines that clean and conform source data first.

Query workloads that aggregate large volumes across many dimensions for reporting and dashboards, favoring scan throughput and columnar layouts over single-row lookups.

A visual analytics artifact combining charts, filters, and metrics over a dataset, refreshed on a schedule or live connection to support business monitoring.