Cloudera Machine Learning Engineer cheat sheet

A Cloudera AI project bundles code, artifacts, jobs, sessions, models, and experiments; it is the unit of collaboration and access control, not a single script.

A session is an interactive compute container launched from a project runtime; the runtime pins Python version plus preinstalled libraries, so dependency drift is controlled by switching runtimes rather than editing the base image.

Project roles map to capabilities: read-only reviewers can view files and results, collaborators can run sessions and jobs, and owners can manage membership and delete the project.

Engine profiles (CPU/GPU/memory) are selected per session or job; the runtime image and the engine profile are independent choices, and mismatches (e.g., GPU code on a CPU profile) fail at runtime, not at launch.

Files created outside the project home directory (e.g., /tmp) do not persist across session restarts; only the project filesystem and mounted external storage survive.

Version control integration: a project can be linked to a Git remote, and sessions can push/pull; uncommitted work in a session is not protected by Git until committed and pushed.

Environment variables and project-level secrets are injected into sessions and jobs at launch, so credentials should never be hardcoded in tracked files.

A job is a non-interactive execution of a script with a schedule or trigger; it uses the same project files as sessions, so code must be runnable without interactive state.

Deleting a project removes its associated jobs, sessions, and models; export or commit reusable assets before deletion.

Hive tables are queried through Impala or Hive on the cluster; use Impala for low-latency interactive SQL on Parquet/Kudu and Hive for batch and transactional workloads.

Kudu suits row-level updates and upserts with primary keys; Parquet is append/overwrite-oriented and better for large scans. Choose Kudu when the pipeline needs mutable records.

HDFS paths use hdfs://namenode:port/path; local project paths are relative to the project root. Mixing them silently reads the wrong file if the scheme is omitted.

Impala metadata is refreshed with INVALIDATE METADATA (schema/table changes) or REFRESH (new data files only); stale metadata causes missing rows or unknown-table errors.

Spark reads from Hive via the metastore when Hive support is enabled; without it, Spark sees only the filesystem and cannot resolve table names.

Partition pruning requires filtering on the partition column in the WHERE clause; wrapping the column in a function (e.g., substr) disables pruning.

Small files degrade scan performance; compact before training. Target file sizes in the hundreds of MB range for Parquet.

Data leakage guard: fit imputers, scalers, and encoders on the training split only, then apply the fitted transformer to validation and test splits.

Schema drift detection compares incoming column names and types against the expected schema; fail the job loudly rather than coercing silently.

An experiment groups runs; each run records parameters, metrics, artifacts, and the code version, enabling comparison across hyperparameter settings.

Log metrics per epoch or per fold, not only the final value; curves reveal overfitting and unstable training that a single number hides.

Cross-validation reduces variance in the performance estimate; use stratified folds for classification with class imbalance and grouped folds when records share an entity.

For time-ordered data, use forward-chaining splits; random splits leak future information into training.

Track the evaluation metric that matches the deployment decision (e.g., recall at a fixed precision for fraud), not just accuracy.

Baseline first: a majority-class or simple linear baseline sets the floor; a complex model that fails to beat it is not justified.

Reproducibility requires recording the random seed, library versions, data snapshot identifier, and runtime image alongside the run.

Feature importance from tree ensembles is biased toward high-cardinality features; use permutation importance on held-out data for a fairer ranking.

Promote a run to a registered model only after it beats the current production model on the same held-out evaluation set.

A model deployment creates a REST endpoint backed by a model artifact plus a runtime and a function that defines the predict logic.

The predict function receives the parsed request body and must return a JSON-serializable object; exceptions surface to the caller as HTTP errors.

Deployment access is controlled by project membership; external callers authenticate with an API key passed as a bearer token.

Scaling is set by the number of replicas; each replica loads the model into memory, so memory per replica bounds the model size you can serve.

Deployment builds are immutable snapshots; changing the model artifact or predict function requires creating a new build and promoting it.

Model input schema validation belongs inside the predict function; reject malformed payloads with a clear error rather than letting the model fail obscurely.

A/B or shadow testing compares a new build against the incumbent by routing a share of traffic before full promotion.

Batch scoring is better served by a job that reads a table and writes predictions than by hammering the REST endpoint row by row.

Jobs support cron-style schedules; the schedule timezone must be explicit, and overlapping runs are avoided by setting a timeout or concurrency limit.

Failed job runs retain logs on the run history; alerting should key on run status transitions rather than on log scraping.

Model drift monitoring compares live input distributions against the training distribution; a shift in feature statistics precedes accuracy decay.

Governance requires lineage: record which data snapshot, code commit, and runtime produced each deployed model so a prediction can be traced back.

Access to sensitive training data should be mediated by project membership and cluster-level authorization; the ML runtime does not bypass Ranger or Sentry policies.

Retire models explicitly: unpromote the deployment and archive the artifact so stale endpoints are not silently called.

Cost control: GPU engine profiles are the dominant spend; stop idle sessions and right-size replicas to observed request volume.

Audit logs record who launched jobs, changed deployments, and accessed data; retain them per the organization's compliance window.

Health checks should include a canary request to the deployment endpoint, not just process liveness, because a loaded model can fail on real payloads.

Project filesystem persists across sessions; /tmp and in-memory state do not.

Runtime image and engine profile are chosen independently at launch.

Roles: viewer (read), collaborator (run), owner (manage membership).

Jobs reuse project files but run without an interactive session.

Git-linked projects require commit and push to protect work.

Secrets are injected as environment variables at launch.

Deleting a project cascades to its jobs, sessions, and models.

Impala: INVALIDATE METADATA for schema changes, REFRESH for new files.

Kudu for upserts and mutable rows; Parquet for large append-only scans.

Spark needs Hive support enabled to resolve metastore table names.

Partition pruning fails when the partition column is wrapped in a function.

Fit transformers on the training split only to avoid leakage.

Compact small files before training; target hundreds of MB per Parquet file.

Always specify the scheme (hdfs:// vs relative project path).

A run records params, metrics, artifacts, and code version; an experiment groups runs.

Time-ordered data needs forward-chaining splits, not random splits.

Stratify folds for class imbalance; group folds by entity to prevent leakage.

Log per-epoch metrics to expose overfitting hidden by final values.

Record seed, library versions, data snapshot, and runtime for reproducibility.

Permutation importance on held-out data beats impurity-based importance for high-cardinality features.

Promotion requires beating the incumbent on the same held-out set.

Deployment = model artifact + runtime + predict function exposed as REST.

predict receives parsed body and must return JSON-serializable output.

Callers authenticate with an API key as a bearer token.

Replicas scale horizontally; per-replica memory bounds model size.

Builds are immutable; changes require a new build and promotion.

Validate input schema inside predict and return clear errors.

Batch scoring belongs in a job, not repeated REST calls.

Cron schedules need an explicit timezone; cap concurrency to prevent overlap.

Alert on run status transitions, not log content.

Drift monitoring compares live feature distributions to training distributions.

Lineage ties each model to data snapshot, code commit, and runtime.

Ranger/Sentry policies still govern data access from ML runtimes.

Unpromote and archive retired models to prevent stale calls.

Canary requests, not liveness checks alone, validate deployment health.