Data services, storage, and workload operations
Domain 3 tests service selection, the object-storage-first storage model, and the operational levers that separate compute from data. Questions often describe a workload pattern and ask which engine or scaling control fits.
Choosing a service
Impala suits low-latency ANSI SQL and BI concurrency; Hive on Tez suits batch ETL and heavy transforms; Spark handles general processing, streaming ingestion, and ML feature work; Kudu (Operational Database) supports upserts and row-level updates for operational apps; HBase serves sparse random reads; Kafka carries event streams; NiFi (DataFlow) moves and transforms data flows. Cloudera Data Warehouse exposes Hive and Impala virtual warehouses; Cloudera Data Engineering runs Spark jobs and sessions on autoscaling virtual clusters; Cloudera Machine Learning hosts ML workloads.
Storage model
Object storage is the default filesystem, addressed as s3a:// on AWS, abfs:// on Azure Data Lake Storage Gen2, and gs:// on GCS. The Data Lake root bucket holds the managed warehouse and service data; a separate logs bucket collects cluster logs. Use lifecycle rules, versioning, and server-side encryption. Local disks are scratch space and the backing store for Kudu, not a durable data layer. There is no HDFS in public cloud, so prefer external tables over managed tables you cannot relocate.
Workload operations
Virtual warehouses and virtual clusters scale and suspend independently of the Data Lake, which is how you isolate BI concurrency from nightly ETL. Data Hub autoscaling reacts to YARN demand; stopping a Data Hub releases compute while data in object storage persists.
Worked example
Thirty concurrent dashboard users plus a nightly ETL job. Reasoning: run two Impala virtual warehouses, an always-on larger one for BI and a smaller auto-suspend one for ad hoc queries, plus a Hive virtual warehouse for ETL, all reading the same Data Lake metastore. A single shared warehouse lets the ETL saturate BI latency, and hosting dashboards on a Data Hub running Kafka wastes resources.
Traps
Assuming Kudu stores data on object storage; using the s3:// scheme where s3a:// is expected; building interactive BI on Hive/Tez; leaving a warehouse always on for occasional queries; believing that deleting a cluster deletes its bucket contents.