Cloudera Data Operator flashcards
Cloudera Data Operator · Build recall with a focused review session.
32 cards. Read the term and the definition here. The deck at the top of this page is optional.
- Cloudera Manager default UI port
- The Cloudera Manager admin web console listens on TCP port 7180 by default; agents communicate with the server on port 7182, and the embedded PostgreSQL database uses 7183.
- Cloudera parcel activation
- A parcel is a Cloudera Runtime binary distribution that must be downloaded, distributed to hosts, and activated before its services can start; multiple parcel versions can coexist on disk.
- NameNode HA JournalNodes
- HDFS high availability uses an active and standby NameNode sharing an edit log through JournalNodes; an odd number of at least three JournalNodes is required for quorum.
- YARN minimum allocation
- YARN allocates container memory in multiples of yarn.scheduler.minimum-allocation-mb, default 1024 MB, so a request below that value is rounded up to one unit.
- Kerberos keytab per service
- Enabling Kerberos on a CDP cluster requires a principal and keytab for each service; services must be restarted after Kerberos is enabled so they load their keytabs.
- Host Inspector purpose
- Cloudera Manager's Host Inspector and cluster validation checks surface version mismatches, DNS problems, and missing Kerberos principals before a service start is attempted.
- CML session versus job
- In Cloudera Machine Learning, a session is an interactive compute pod for exploration, while a job is a scheduled or on-demand non-interactive run of a script or pipeline.
- CML model deployment endpoint
- A CML model deployment exposes a trained model as a REST endpoint inside the cluster; it records model metadata for governance and can be promoted through approval stages.
- CML experiment tracking
- A CML experiment is a tracked run that logs parameters, metrics, and artifacts, letting data scientists compare model versions and reproduce results across iterations.
- Custom engine image
- A CML custom engine is a container image built with extra libraries and pushed to the cluster registry; projects select it as the runtime for sessions and jobs.
- RAG pipeline components
- A retrieval-augmented generation pipeline pairs an embedding model and a vector index with a generative LLM, injecting retrieved context into the prompt at inference time.
- GPU node pool requirement
- GPU-backed CML sessions require node pools with GPU instances; without them, sessions fall back to CPU engines and deep learning workloads run far slower.
- Kafka partition ordering
- Kafka guarantees message order only within a single partition; a topic with multiple partitions offers no global ordering, so keys are used to pin related records together.
- Consumer group partition limit
- Within a consumer group each partition is assigned to at most one consumer, so useful parallelism is capped at the topic's partition count; extra consumers stay idle.
- Kafka acks=all durability
- With acks=all the producer waits for all in-sync replicas to acknowledge; min.insync.replicas sets how many replicas must be in sync for the write to succeed.
- Log compaction retention
- Setting cleanup.policy=compact retains the latest record per key rather than deleting by age, which suits changelog and state topics that need current values only.
- Schema Registry compatibility modes
- Schema Registry compatibility modes include BACKWARD, FORWARD, FULL, and transitive variants; the mode on a subject determines which new schemas are accepted.
- Kafka Streams exactly-once
- Exactly-once processing in Kafka Streams requires processing.guarantee=exactly_once_v2 plus a transactional producer and consumers reading with isolation.level=read_committed.
- Kafka Connect task parallelism
- A Kafka Connect connector splits work into tasks, the unit of parallelism; converters such as Avro or JSON Schema determine how records are serialized to topics.
- Iceberg snapshot time travel
- Each Iceberg write creates an immutable snapshot recorded in metadata; time travel queries read a prior snapshot by ID or timestamp without copying the underlying data.
- Iceberg hidden partitioning
- Iceberg stores partition values via transforms such as bucket, truncate, day, and hour, so queries filter on source columns without referencing partition columns explicitly.
- Iceberg partition evolution
- Iceberg allows changing a table's partition spec without rewriting existing files; each snapshot records its own spec, and old data keeps its original layout.
- Hive ACID table requirements
- Hive ACID transactional tables require ORC format and bucketing; delta files record changes and background compaction merges them into base files for read performance.
- Delta Lake transaction log
- Delta Lake records commits as JSON entries in the _delta_log directory, with Parquet checkpoints; readers reconstruct table state from the log for consistent snapshots.
- Delta OPTIMIZE and Z-ORDER
- Delta OPTIMIZE compacts many small files into larger ones, and Z-ORDER co-locates related values so data-skipping statistics let queries read fewer files.
- Kudu upsert workloads
- Apache Kudu is a columnar store with primary keys supporting fast upserts and deletes, often paired with Impala for analytical workloads that need row-level updates.
- Managed versus external table
- Dropping a managed table removes both metadata and data, while dropping an external table removes only metadata and leaves the underlying files in place.
- Atlas lineage and tags
- Apache Atlas captures lineage from Hive, Impala, and Spark jobs, and classifications applied to datasets propagate along lineage edges for downstream policy enforcement.
- Ranger deny precedence
- Ranger evaluates policies per resource and user or group, and a deny policy overrides any matching allow policy, so explicit denials block access even when grants exist.
- Ranger column masking
- Ranger column masking and row filtering are applied by the query engine at read time, so results are redacted consistently without altering the stored files.
- Data fabric virtualization
- A data fabric exposes external tables through a shared catalog and query engine, letting users query remote sources without copying data, trading latency for freshness.
- Data mesh domain ownership
- A data mesh assigns ownership of data products to business domains, while a data fabric supplies the shared catalog, governance, and access layer across those products.