Cloudera Data Operator key terms

Cloudera's integrated platform that combines data warehouse, data engineering, machine learning, and streaming services on a shared security and governance layer, deployable in public cloud or on-premises environments.

Administrative console and control plane used to install, configure, monitor, and troubleshoot clusters and services, including role-level health checks and configuration history across environments.

A CDP cluster template that provisions and manages compute clusters running services such as HDFS, YARN, Hive, and Spark, typically attached to an external or embedded data lake.

A versioned binary distribution of service software that Cloudera Manager installs and activates on hosts, enabling upgrade and rollback without reinstalling the operating system packages.

A single running process of a service on a specific host, such as a NameNode or RegionServer, whose configuration, logs, and health are managed individually by Cloudera Manager.

A configuration in which redundant service roles, such as multiple NameNodes or JournalNodes, coordinate so the cluster survives the loss of a single host or process without data loss.

The CDP framework that centralizes authentication, authorization, auditing, and metadata lineage across clusters and services, so governance policies follow data rather than individual compute environments.

Policy engine that enforces fine-grained access control over Hadoop and related services, storing policies centrally and evaluating them at query time for users, groups, and roles.

Metadata and governance service that tracks technical and business lineage, classifies datasets, and exposes a REST API so auditors and stewards can trace how data was produced and consumed.

A recorded graph of how datasets derive from upstream sources through transformations, used for impact analysis, debugging, regulatory reporting, and root-cause investigation when downstream results look wrong.

A curated catalog of agreed business terms, definitions, and owners linked to physical datasets, helping analysts interpret fields consistently and helping stewards apply the right policies.

A scalable storage layer holding raw and refined data in open file formats, typically with object storage or HDFS underneath, allowing many engines to read the same datasets without copying.

A governed analytical store optimized for structured SQL workloads, where schemas are enforced and performance is tuned for concurrent business intelligence queries rather than raw ingestion.

SQL-on-Hadoop service that maps tables onto distributed storage and compiles queries into execution engines, widely used for batch ETL and as a shared metastore for other engines.

The service that stores table definitions, partitions, and storage locations, acting as a shared catalog so engines like Spark, Impala, and Hive resolve the same tables consistently.

Massively parallel SQL query engine that runs natively on cluster data without MapReduce, designed for low-latency interactive analytics over Hive tables and open file formats.

A precomputed query result stored physically and refreshed on a schedule or incrementally, letting repeated aggregations return quickly while the underlying detail tables keep changing.

Cluster resource manager that allocates CPU and memory to distributed applications, queues their requests, and isolates workloads so batch, streaming, and interactive jobs can share hosts.

A hierarchical YARN queue with configured capacity and maximum limits, used to guarantee resources to teams and prevent one workload from starving others on a shared cluster.

Distributed in-memory processing engine for batch and streaming workloads, offering DataFrames, SQL, and machine learning libraries that run on YARN, Kubernetes, or standalone clusters.

Spark's incremental processing model that treats a live data stream as an unbounded table, supporting event-time windows, watermarks, and exactly-once sinks for continuous pipelines.

Distributed commit log for publishing and subscribing to event streams, partitioning topics across brokers and retaining records so multiple consumers can replay independently.

An ordered, append-only shard of a Kafka topic that provides parallelism and ordering guarantees within itself, with replicas spread across brokers for fault tolerance.

A set of Kafka consumers that cooperatively divide partition assignments for a topic, so each partition is processed by one member while overall throughput scales with group size.

The act of recording a consumer group's progress position per partition, enabling resumption after restart and determining whether processing semantics are at-least-once or effectively once.

A service that stores and versions message schemas for streaming topics, letting producers and consumers negotiate compatibility rules and reject incompatible payload changes before they break pipelines.

A processing guarantee where each event affects the output state precisely one time, achieved through idempotent writes, transactional commits, and coordinated checkpoints across sources and sinks.

Periodically persisting stream processing state and source positions to durable storage, so a failed job can restart from a consistent point and recover its accumulated aggregations.

Stream processing engine built around continuous operators and event time, offering low-latency stateful computation, savepoints, and exactly-once guarantees for high-volume event pipelines.

Computing results based on the timestamp embedded in each event rather than arrival time, using watermarks to decide when late data can be dropped or results finalized.

A moving threshold in a streaming job that signals how far event time has advanced, allowing windowed aggregations to emit results while tolerating a bounded amount of late data.

A table specification layered over files in object storage that adds transactions, schema evolution, and time travel, so multiple engines share one governed copy of the data.

Open table format using manifest files and snapshots to provide ACID commits, hidden partitioning, and snapshot isolation on data lake storage without rewriting existing files.

Open table format and storage layer that adds a transaction log to Parquet files, enabling ACID updates, schema enforcement, and time travel for lakehouse workloads.

A concurrency model where readers see a consistent table state as of a point in time while writers commit new snapshots, preventing read queries from blocking ingestion jobs.

The ability to query a table as it existed at a prior snapshot or timestamp, useful for auditing, reproducing past reports, and recovering from accidental writes.

Safely changing a table's columns, such as adding or renaming fields, without rewriting all data, while keeping readers compatible through defined compatibility rules.

Durable, flat-namespace storage accessed by key over HTTP, such as S3 or Ozone, commonly used as the lakehouse foundation because it scales cheaply and decouples storage from compute.

A searchable inventory of datasets, schemas, owners, and classifications that helps users find trustworthy data and helps administrators apply consistent access and retention policies.

An architectural approach that connects distributed data sources through shared metadata, governance, and access interfaces, presenting consistent, policy-controlled data to consumers regardless of where it physically resides.

Querying remote sources through a logical layer without copying their data, letting users join across systems while governance and performance tuning happen at the federation layer.

Granting other teams or environments read access to datasets through metadata and permissions rather than duplicating files, reducing storage cost and keeping a single governed source of truth.

A policy that dynamically obscures sensitive field values for unauthorized users at query time, such as showing partial account numbers, while privileged roles see full values.

A policy that restricts which records a user can retrieve based on their identity or group, so different users querying the same table see different subsets of rows.

The number of copies of each data block or partition kept across hosts or brokers, trading storage cost for durability and read availability when a node fails.