Cloudera Machine Learning Engineer flashcards

32 cards. Read the term and the definition here. The deck at the top of this page is optional.

Project persistence scope

Only the project home filesystem survives session restarts. Files written to /tmp or held in memory are lost, so persist outputs and intermediate artifacts under the project directory.

Runtime versus engine profile

The runtime fixes the OS image and preinstalled libraries; the engine profile fixes CPU, memory, and GPU. They are selected independently, and GPU code on a CPU profile fails only at execution time.

Project role capabilities

Viewers read files and results, collaborators launch sessions and jobs, and owners additionally manage membership and delete the project. Roles gate actions, not data-level row filtering.

Session versus job

A session is interactive and stateful for exploration. A job runs a script non-interactively on a schedule or trigger, sharing project files but with no interactive state to rely on.

Git-linked project protection

Linking a project to a Git remote does not auto-commit work. Uncommitted session changes are unprotected until committed and pushed, so commit before risky refactors.

Secrets injection at launch

Project-level secrets and environment variables are injected into sessions and jobs at startup. Credentials should be referenced from these variables, never hardcoded in tracked files.

Project deletion cascade

Deleting a project also removes its associated jobs, sessions, and model deployments. Export artifacts or push code to Git before deletion if anything must be retained.

Impala metadata refresh commands

Use INVALIDATE METADATA after schema or table changes so Impala reloads table definitions. Use REFRESH after new data files land in an existing table to pick up added rows.

Kudu versus Parquet choice

Kudu supports primary keys and row-level upserts, suiting mutable records. Parquet is append- and overwrite-oriented, better for large analytical scans with no in-place updates.

Spark Hive metastore access

Spark resolves Hive table names only when Hive support is enabled. Without it, Spark reads raw filesystem paths and cannot find tables registered in the metastore.

Partition pruning failure mode

Filtering directly on the partition column enables pruning. Wrapping that column in a function such as substr or cast forces a full scan and defeats pruning.

Transformer fitting discipline

Fit imputers, scalers, and encoders on the training split only, then apply the fitted objects to validation and test data. Fitting on all data leaks test information into training.

Small file compaction

Many tiny Parquet files add per-file overhead to scans. Compact them into larger files, typically hundreds of megabytes each, before training to cut read time.

HDFS path scheme requirement

HDFS locations need an explicit hdfs://namenode:port prefix. A bare path resolves relative to the project filesystem, silently reading the wrong data when the scheme is omitted.

Schema drift handling

Compare incoming column names and types to the expected schema at ingestion. Fail the job with a clear message rather than silently coercing types and corrupting downstream training data.

Experiment versus run

An experiment is a named grouping of related runs. Each run records its own parameters, metrics, artifacts, and code version so runs can be compared side by side.

Forward-chaining validation

For time-ordered data, train on the past and validate on later periods, advancing the window each fold. Random splits leak future information into training and inflate scores.

Stratified and grouped folds

Stratified folds preserve class proportions for imbalanced targets. Grouped folds keep all records of one entity in a single fold, preventing leakage across train and validation.

Per-epoch metric logging

Logging metrics at each epoch or fold exposes training curves. A rising validation loss while training loss falls indicates overfitting that a single final number conceals.

Reproducibility record contents

Record the random seed, library versions, data snapshot identifier, and runtime image with each run. Without these, a result cannot be reproduced or audited later.

Permutation importance rationale

Impurity-based tree importance favors high-cardinality features. Permutation importance measured on held-out data by shuffling each feature gives a fairer, model-agnostic ranking.

Baseline before complexity

Establish a majority-class or simple linear baseline first. A complex model that fails to beat the baseline on the same evaluation set is not justified by its added cost.

Deployment composition

A deployment combines a stored model artifact, a runtime image, and a predict function, exposed as a REST endpoint. Each replica loads the artifact into memory.

Predict function contract

The predict function receives the parsed request body and must return a JSON-serializable object. Uncaught exceptions surface to the caller as HTTP error responses.

Endpoint authentication method

External callers authenticate to a model endpoint with an API key sent as a bearer token. Access also depends on the caller having project membership rights.

Replica scaling limits

Replicas scale horizontally to handle more requests. Each replica holds a full model copy in memory, so per-replica memory limits the largest model you can serve.

Immutable deployment builds

A deployment build is an immutable snapshot of artifact and predict logic. Changing either requires creating a new build and promoting it, which allows rollback to the prior build.

Input validation in predict

Validate the payload schema inside the predict function and reject malformed requests with a clear error. Letting the model fail on bad input produces opaque errors for callers.

Batch scoring via jobs

Score large datasets with a job that reads a table and writes predictions. Calling the REST endpoint row by row adds network overhead and is far slower.

Job schedule timezone

Cron schedules must specify an explicit timezone. Pair the schedule with a run timeout or concurrency limit so a slow run does not overlap the next scheduled start.

Drift monitoring signal

Compare live input feature distributions against the training distribution. A statistically significant shift in inputs typically precedes measurable decay in prediction accuracy.

Model lineage requirement

Lineage links each deployed model to its data snapshot, code commit, and runtime image. This lets an auditor trace any prediction back to the exact assets that produced it.