I85PREP
⌘ K
Cloudera CDP Generalist study materials
Cloudera CDP Generalist study resources

Cloudera CDP Generalist: CDP Architecture and Deployment Models

Cloudera CDP Generalist · Work through your exam, one topic at a time.

All guide sections

CDP Architecture and Deployment Models

This domain checks whether you can place any CDP workload in the right delivery model and state which components are shared versus per-cluster. Questions give a constraint - on-premises data residency, elastic SQL, Kubernetes already in place - and expect you to name the model and the services involved.

The three delivery models

  • CDP Private Cloud Base: the direct successor to CDH. Installed on physical hosts or VMs with Cloudera Manager, ships Cloudera Runtime 7.x as parcels, keeps data in HDFS or Ozone on the same nodes, and runs work on YARN.
  • CDP Private Cloud Data Services: the same Runtime base cluster plus a Kubernetes layer (Red Hat OpenShift) hosting containerised data services - Data Warehouse, Data Engineering, Machine Learning, DataFlow, Operational Database. Compute scales independently; storage can be Ozone or an external S3-compatible store.
  • CDP Public Cloud: Cloudera's control plane provisions an environment on AWS, Azure or GCP. Each environment holds one Data Lake, any number of Data Hubs (classic clusters), and data services on the provider's Kubernetes. Storage is the cloud object store, not HDFS.

Shared versus per-cluster

The Data Lake hosts the SDX services: Ranger, Atlas and the Hive Metastore; in public cloud it also hosts IDBroker, FreeIPA and the DSSD object store. Data Hubs and data services attach to that Data Lake and do not run their own Ranger or Metastore. This is the single most examined architectural fact.

Worked example

A bank must keep data in its own data centre, already runs OpenShift, and wants analysts to spin up SQL warehouses that idle when unused. Private Cloud Base is rejected: without Kubernetes there is no elastic, suspendable compute. Public Cloud is rejected on data residency. Answer: CDP Private Cloud Data Services, with an Ozone-backed base cluster and CDW virtual warehouses on OpenShift.

Traps

  • Assuming every CDP cluster has HDFS - public-cloud data services read object storage.
  • Saying Ranger is installed per Data Hub; it lives in the Data Lake.
  • Treating a Data Hub as part of the Data Lake rather than a consumer of it.
  • Forgetting the control plane exists only in CDP Public Cloud.
  • Mixing up IDBroker (credential vending) with FreeIPA (Kerberos/LDAP/DNS).

Data Lake, SDX, Security and Governance

This domain tests the shared security and governance layer, SDX, and how authorisation actually flows from a written policy to a running query. Expect scenarios about who can see which column, how lineage is captured, and where a tag-based policy takes effect.

What SDX contains

The Hive Metastore is the single catalog for Hive, Impala, Spark and CDW; a table created through one engine is visible to all. Ranger is the policy decision point for HDFS, Hive/Impala, HBase, Kafka, YARN and Ozone, and for object-store buckets in public cloud. Atlas stores technical metadata, lineage and classifications. Encryption uses Ranger KMS for HDFS transparent data encryption. In public cloud, FreeIPA supplies Kerberos and LDAP and IDBroker vends short-lived cloud credentials so workloads never hold long-lived keys.

How a Ranger policy evaluates

A policy binds a resource (database, table, column, URL, path, topic) to users, groups or roles with permissions (select, insert, create, alter), optional conditions such as IP range or time window, and optional data-hiding rules - column masking and row filtering, supported for Hive and Impala. Tag-based policies reference an Atlas classification instead of a named resource, so one policy covers every table tagged PII.

Worked example

An analyst needs every row of customers, but card numbers must show only the last four digits. A row filter is wrong: it removes rows. Create a Ranger masking policy on the Hive service for database sales, table customers, column card_no, group analysts, using a mask that keeps the last four characters - or tag the column PCI in Atlas and write one tag policy that all similarly tagged columns inherit. Atlas lineage then shows the query and its downstream tables because the Hive/Impala hooks report it.

Traps

  • Naming Sentry or Navigator as CDP components; both belong to CDH. CDP uses Ranger and Atlas.
  • Granting a Ranger HDFS policy and assuming it covers S3 in public cloud - object-store access needs its own policy.
  • Expecting tag policies to work without the Atlas tag being synced to Ranger.
  • Confusing masking (hides values, keeps rows) with row filtering (drops rows).
  • Believing each Data Hub owns its own Metastore; that assumption breaks cross-engine queries.

Data Warehousing and SQL Analytics

This domain covers how CDP answers SQL questions: which engine fits a workload, how virtual warehouses differ from YARN-based Impala, and what the Database Catalog actually is. Questions are typically "pick the engine" or "explain why this query is slow or fails".

Engines and their contracts

  • Impala: MPP engine with its own daemons; it does not run on YARN or Tez. Low latency and high concurrency; reads Parquet, ORC, Avro, text, Kudu, HDFS, object storage and Kafka. A node failure kills a running query, so it is not a batch workhorse.
  • Hive on Tez: batch engine on YARN, fault tolerant, supports ACID transactions and full DML. Right for large ETL, wrong for interactive concurrency.
  • Kudu: mutable columnar store with primary keys, used for upserts, point lookups and reporting over changing data; weaker than Parquet for full-table scans.
  • CDW: a Database Catalog backed by the Data Lake Metastore plus virtual warehouses - isolated Impala or Hive compute that can autoscale and auto-suspend. Isolation per warehouse is how you separate workloads and cost.

Worked example

Two teams share one dataset: BI runs 40 concurrent dashboards over Parquet in object storage, while ETL rebuilds aggregates nightly. A single shared Impala cluster lets the ETL starve the dashboards. Answer: two virtual warehouses - a smaller Impala warehouse with autoscaling for BI, a larger Hive warehouse for the nightly job - both pointed at the same Database Catalog so they see identical tables.

Traps

  • Saying Impala runs on YARN or uses Tez; it does neither.
  • Assuming every engine that reads Hive tables can also write ACID tables; verify support instead of assuming parity.
  • Thinking a warehouse is a physical database; it is compute plus a catalog namespace.
  • Forgetting auto-suspend: idle warehouses stop, so the first query afterwards is slower.
  • Blaming storage for slow joins actually caused by missing statistics or poor partitioning; COMPUTE STATS and ANALYZE TABLE still matter.

Data Engineering, Streaming and Operational Data

This domain tests moving and processing data: YARN and Spark on classic clusters, CDE virtual clusters, NiFi flows, Kafka topics, and operational stores such as HBase/Phoenix and Ozone.

Batch and scheduling

On clusters, Spark runs on YARN, where queue placement, memory overhead and dynamic allocation drive most failures. In CDP Public Cloud and Private Cloud Data Services, Cloudera Data Engineering provides virtual clusters - isolated Spark on Kubernetes - with jobs, sessions, dependencies and Airflow-based orchestration. Jobs are the scheduled, retryable unit; sessions are interactive and consume resources until stopped.

Streaming and flow

NiFi handles flow-based ingestion: processors, connections with backpressure thresholds, parameter contexts for environment-specific values, and provenance records that let you replay a specific event. Kafka provides a durable partitioned log; partition count caps consumer parallelism, and replication factor sets durability. Streams Messaging Manager lets you inspect topics and consumer lag without writing code.

Operational stores

HBase gives random read/write on a sorted row key; Phoenix adds SQL over it, and Cloudera Operational Database packages both as a service. Ozone is the S3-compatible object store: volumes and buckets above a key namespace, designed to remove the NameNode scale ceiling of HDFS.

Worked example

A team must load Kafka events into Kudu every minute with no duplicates after a restart. Structured Streaming alone does not deduplicate across restarts: you need a durable checkpoint location plus an idempotent write. Because Kudu has a primary key, use upsert mode keyed on event id, so a replayed batch overwrites rather than duplicates. NiFi would be the right tool if the requirement were only routing and delivery, not stateful windowed aggregation.

Traps

  • Designing more consumers than partitions and expecting extra parallelism.
  • Treating NiFi provenance as long-term storage; it is bounded by size and age settings.
  • Assuming an HDFS path and an Ozone path are interchangeable without changing the URI scheme.
  • Treating a CDE session as free - it holds compute until explicitly stopped.
  • Forgetting that Spark on YARN needs executor memory overhead, not just executor memory.

Machine Learning and Model Deployment

This domain covers the applied machine learning lifecycle in CDP: where code runs, how it reaches governed data, and how a model becomes a served endpoint. Questions are situational - pick the right CML construct for a described need.

Building blocks

CML runs on Kubernetes as a data service, organised into workspaces (a namespace with its own resource quota, members and permissions) and projects (git-backed, shared code and artefacts). Inside a project you get sessions (interactive Jupyter, JupyterLab or SSH with a chosen engine and resource profile), jobs (scheduled or on-demand script runs with history and retries), models (artefacts registered from a run) and deployments (served endpoints for real-time or batch scoring). Accelerators are prebuilt project templates for common use cases.

Access to governed data

CML does not bypass SDX. It reads through the Data Lake catalog, and Ranger still enforces table and column permissions. Connections use stored data connections plus the environment's Kerberos and IDBroker credentials; embedding long-lived cloud keys in a project is both a security failure and an exam trap.

Worked example

A model must be retrained nightly and served to an application with low latency. Three constructs are needed, not one: a job on a nightly schedule to retrain, a model registered from that job's run, and a deployment with a fixed replica count serving the endpoint. The interactive session is only for development and should be stopped - a resident session is the common wrong answer, because it consumes quota continuously and exposes no stable endpoint.

Traps

  • Choosing a session for production serving, or a deployment for exploratory work.
  • Assuming workspace membership grants data access; Ranger still decides table permissions.
  • Forgetting quotas are per workspace, so a heavy job can starve colleagues' sessions.
  • Assuming a deployed model retrains itself; a deployment serves the artefact you registered.
  • Ignoring engine and library version pinning, which lets a retrain silently change dependency versions.

Operations, Migration and Disaster Recovery

This domain tests running CDP on day two: Cloudera Manager mechanics, upgrade and migration paths, and how Replication Manager delivers disaster recovery. Questions are framed as "what is the least-disruptive way" or "what is protected and what is not".

Cloudera Manager and Runtime

Cloudera Manager deploys Cloudera Runtime as parcels - versioned tarballs that are downloaded, distributed, activated on hosts, and only then followed by service restarts - rather than as OS packages. Configuration changes mark services "stale"; a restart applies them. Role groups let one configuration apply to many role instances, which is the efficient way to change a setting across a cluster.

Upgrade and migration paths

Two routes exist. In-place: upgrade the existing cluster through Cloudera Manager to Cloudera Runtime 7, which requires a supported OS, JDK and a valid CDH baseline, and which converts Sentry privileges into Ranger policies. Side-by-side: stand up a new CDP cluster, replicate data and metadata with Replication Manager, cut over, then decommission. Side-by-side is the usual answer when the source OS or CDH version is out of the supported path, or when downtime must be near zero.

Disaster recovery with Replication Manager

Replication Manager, the successor to BDR, runs snapshot-based replication policies on a schedule - HDFS, Hive (data plus Metastore metadata), HBase, Ozone, and Ranger policies as a distinct policy type - and can replicate to cloud object storage. Because each policy carries its own schedule, the effective RPO is set per dataset rather than globally.

Worked example

A requirement states RPO of one hour for a Hive warehouse, and that authorisation must survive failover. One hourly Hive replication policy covers the tables and their Metastore entries; a second, separate Ranger policy replication covers authorisation. Scheduling only the Hive policy leaves the DR cluster readable but unsecured - a classic partial answer.

Traps

  • Assuming Replication Manager covers every service; Kudu and Kafka are not replicated by it.
  • Treating Hive replication as a file copy - without Metastore replication the tables are unreadable at the target.
  • Forgetting backups of Cloudera Manager's own database and the Ranger and Metastore databases.
  • Expecting a downloaded parcel to take effect automatically; activation and restarts are required.
  • Believing Sentry policies migrate silently; converted rules, especially URI-based ones, must be validated.