I85PREP
⌘ K
Cloudera Data Operator study materials
Cloudera Data Operator study resources

Cloudera Data Operator: Open Data Lakehouse on CDP

Cloudera Data Operator · Work through your exam, one topic at a time.

All guide sections

Open Data Lakehouse on CDP

The Open Data Lakehouse domain tests Cloudera's lakehouse design on CDP: open table formats, ACID transactions, separation of storage and compute, and the roles of CDW and CDE. Questions probe Iceberg versus Hive ACID, performance tuning, and governance across zones.

Core concepts

  • A lakehouse combines data lake storage (object stores such as S3 or ADLS, or HDFS) with warehouse-like ACID, schema, and performance. CDP supports Apache Iceberg and Hive ACID. Iceberg is the strategic open table format for hidden partitioning, time travel, schema and partition evolution, and metadata tables.
  • CDW provides Impala and Hive for BI and SQL with isolated virtual warehouses, auto-scaling, and workload management. CDE provides Spark for ETL/ELT with jobs, pipelines, and dependencies.
  • Storage and compute are separated: data lives in object storage; compute clusters are ephemeral and sized per workload. This enables independent scaling and cost control.
  • Governance: Ranger provides fine-grained access, row filters, and column masking; Atlas provides lineage and classification; SDX shares context.
  • Performance: optimize file sizes (avoid small files), partition or bucket appropriately (Iceberg hidden partitioning), use statistics, compact Iceberg tables, and avoid over-partitioning.

Worked example

A retailer needs daily ETL and BI with time travel for audits. Raw events land in S3; CDE Spark reads JSON, deduplicates, and writes an Iceberg curated table partitioned by day; CDW Impala queries the Iceberg table for dashboards; auditors query AS OF a timestamp. Reasoning: CDE handles Spark ETL; Iceberg provides ACID, time travel, and schema evolution; CDW gives high-concurrency SQL; Ranger masks PII for analysts.

Traps

  • Assuming Iceberg requires a proprietary catalog; it uses open catalogs such as Hive Metastore or Nessie.
  • Confusing Hive ACID with Iceberg; both provide ACID but differ in features and ecosystem.
  • Ignoring small-file problems; they degrade query performance.
  • Thinking CDW only queries Hive tables; it can query Iceberg and other formats.
  • Over-partitioning by high-cardinality columns; use Iceberg hidden partitioning or bucket transforms.
  • Forgetting that time travel retention depends on snapshot expiration settings.