Cloudera CDP-3002: Cloudera Data Platform Operations and Troubleshooting

The Operations and Troubleshooting domain tests how you monitor a CDP cluster, diagnose service failures, and recover safely. Questions target Cloudera Manager health tests, logs, metrics, HDFS safe mode, YARN scheduling, disk pressure, and upgrade hygiene.

Monitoring surfaces

• Cloudera Manager Home, Charts, Health Tests, Events, and Alerts are the first diagnostic stops. Health test failures often identify a role or dependency before users report an outage.
• Role logs live under service-specific log directories; Cloudera Manager Server logs are separate. Diagnostic bundles collect logs and configuration for support.
• Watch HDFS capacity, missing blocks, and safe mode; YARN pending containers and queue usage; Impala memory; and JVM garbage collection.

Common failure patterns

• HDFS safe mode occurs during NameNode startup, low disk thresholds, or manual entry. Check hdfs dfsadmin -safemode get and -report before forcing anything.
• A YARN application stuck in ACCEPTED is usually waiting for scheduler capacity or queue limits, not failing in Spark. Check queue status and cluster memory.
• A YARN container killed for memory needs more executor memory overhead, not a NodeManager restart.
• Disk full on a DataNode can block writes; check reserved space and log growth, then rebalance.

Worked example

A Spark job remains ACCEPTED for an hour. Reasoning: open the ResourceManager UI and check the queue’s used and maximum capacity, user limits, and pending containers. If the queue is full, wait or adjust Capacity Scheduler configuration; restarting NodeManagers does not create capacity and destroys diagnostic state. If containers later fail with memory errors, increase executor memory overhead.

Traps

• Restarting services before capturing logs and health-test output.
• Deleting logs or audit records to free space.
• Treating safe mode as proof of corruption; it is often a protective state.
• Ignoring health-test failures during parcel upgrades.
• Overlooking clock skew or DNS when Kerberos authentication fails.