NVIDIA NCP-AII study plan

Map the physical and logical layout of a GPU server: PCIe switch topology, GPU-to-NIC affinity, CPU sockets, DIMM channels, and PSU redundancy zones

Explain how to verify GPU enumeration and link width/speed using nvidia-smi queries and lspci output, including what a degraded x8 or Gen3 link implies for training throughput

Describe the installation sequence constraints for GPUs and accelerators: torque specs, air baffles, riser placement, and why power sequencing order matters

Identify the firmware and BIOS settings that must be aligned before GPU install (Above 4G decoding, resizable BAR, PCIe ACS, IOMMU) and state the failure symptom for each when misconfigured

Install the NVIDIA driver, CUDA toolkit, and container toolkit on a supported Linux distribution, and verify each layer independently before moving to the next

Diagnose driver/framework version mismatches by reading the CUDA compatibility matrix and explaining forward compatibility versus minor version compatibility

Configure GPU persistence mode, compute mode, and MIG partitions, and justify when MIG hurts versus helps a given workload

Set up NVIDIA Container Toolkit and run a GPU-enabled container with the correct runtime flags and device visibility

Compare InfiniBand and RoCEv2 for AI fabric design on latency, congestion control, and operational complexity, and pick a fabric for a stated cluster size and budget

Explain GPUDirect RDMA and the role of the GPU-to-NIC PCIe path in collective performance

Describe rail-optimized topology and why rail alignment between GPU index and NIC port matters for AllReduce

Configure and verify IPoIB or RoCE interfaces, MTU, and PFC/ECN settings on a switch and host pair

Run and interpret an NCCL bandwidth test across nodes, and explain how to isolate whether a low result is fabric, PCIe, or software configuration

Trace a slow AllReduce from NCCL debug logs to a specific NIC, rail, or switch port

Compare storage options for AI clusters (local NVMe, NFS, Lustre, parallel filesystems, object storage) on throughput, metadata performance, and checkpoint suitability

Size storage for dataset ingestion and checkpoint writes given a stated model size, checkpoint frequency, and training step time

Deploy a containerized workload scheduler on the cluster and register GPU resources so jobs request the correct number of devices

Configure GPU sharing policies (exclusive, MIG, time-slicing) in the orchestrator and predict scheduling behavior for each

Build a container image for a training job with pinned CUDA, framework, and NCCL versions, and verify it runs identically on two nodes

Explain job preemption, gang scheduling, and queue priority trade-offs for multi-tenant GPU clusters

Instrument a node with GPU, fabric, and health telemetry and define the metrics that predict a failing GPU or degraded link

Design alert thresholds for ECC errors, XID errors, thermal throttling, and NVLink or PCIe replay, and justify each threshold

Execute a structured triage for a simulated XID error and a simulated NCCL hang, stating the first three diagnostic commands in order

Validate a newly installed node with a burn-in sequence covering GPU stress, memory, fabric, and storage

Implement host and fabric security controls: firmware signing, secure boot implications for driver loading, BMC hardening, and fabric partition or tenant isolation

Explain multi-tenant isolation on shared GPUs and the residual risks of time-slicing versus MIG

Design a backup and configuration-drift strategy for drivers, firmware, and cluster config so a node can be rebuilt to a known-good state

Practice reading a support bundle or log set and produce a root-cause statement with evidence

Re-derive the highest-risk facts from memory: driver/CUDA compatibility rules, NCCL tuning levers, fabric congestion settings, and GPU health thresholds

Complete timed practice sets weighted by domain percentage, then review every missed item to its root cause

Close remaining gaps by rebuilding the weakest runbook section from scratch without notes

Rehearse explaining a full cluster bring-up end to end in under five minutes