NVIDIA NCP-AIO study plan
NVIDIA NCP-AIO · Give each week a clear focus.
8 weeks · About 6 hours per week
AI Infrastructure and Data Center Operations & Accelerated Computing and GPU Operations
Map the physical layers of an AI data center (rack power density, liquid vs air cooling, power distribution, floor loading) and explain how GPU TDP and rack-scale systems drive each constraint
Compare GPU, CPU, DPU, and NIC roles in a node and describe how PCIe lanes, NUMA topology, and host memory bandwidth bound end-to-end throughput
Trace the path of one training step from storage read through host memory, PCIe, GPU HBM, and back, naming the bottleneck at each hop
Configure and interpret nvidia-smi, nvidia-smi topo -m, and DCGM basics to read utilization, ECC errors, thermal throttling, and XID events
Your checkpoint
A one-page annotated node diagram plus a written bottleneck analysis of a 3-hop data path (storage to GPU to interconnect), each hop labeled with the metric you would monitor.
AI Infrastructure and Data Center Operations & Accelerated Computing and GPU Operations
Diagnose common GPU failure signatures (XID codes, ECC uncorrectable errors, fallen-off-the-bus, thermal/power capping) and state the first triage action for each
Explain MIG partitioning, GPU sharing, and time-slicing tradeoffs for multi-tenant AI infrastructure
Describe data center operations practices: firmware and driver lifecycle management, canary upgrades, maintenance windows, and change control for GPU fleets
Calculate effective FLOPS, memory-bandwidth roofline limits, and power/cooling headroom for a given GPU count
Your checkpoint
A GPU incident runbook covering five failure signatures with symptom, likely cause, first action, and escalation path.
AI Factory Deployment and Management
Compare reference architectures for AI factory deployment: compute, storage, management, and in-band/out-of-band network fabrics and their isolation requirements
Sequence a bare-metal cluster bring-up from BMC provisioning through OS install, driver/CUDA install, container runtime, and scheduler registration
Explain cluster validation steps (GPU health checks, NCCL all-reduce smoke tests, bandwidth and latency baselines) and define pass/fail thresholds
Describe lifecycle management at fleet scale: image-based provisioning, configuration drift detection, and node drain/return procedures
Your checkpoint
A numbered deployment checklist from bare metal to a validated, scheduler-registered GPU node, with a pass/fail criterion for each step.
AI Factory Deployment and Management & AI Networking and Interconnect
Explain InfiniBand vs RoCEv2 vs Ethernet for AI fabrics and when each is appropriate, including lossless configuration concepts (PFC, ECN, congestion control)
Describe fat-tree and rail-optimized topologies and why rail-aligned NIC-to-GPU mapping matters for collective performance
Interpret fabric health signals: link errors, symbol errors, port down events, and how they correlate with job slowdowns
Map a collective communication pattern (all-reduce, all-gather) onto the fabric and predict which tier (NVLink, intra-rail, spine) carries the traffic
Your checkpoint
A fabric design note for a 64-GPU cluster: chosen topology, cabling map at the rail level, oversubscription ratio, and the three health metrics you would alert on.
AI Networking and Interconnect & Accelerated Computing and GPU Operations
Explain NVLink, NVSwitch, and multi-node interconnect roles and how domain size changes collective scaling behavior
Run and interpret NCCL tests (all-reduce bandwidth and bus bandwidth) at multiple message sizes and identify where scaling degrades
Diagnose stragglers and rank-imbalance effects in distributed training caused by network or GPU heterogeneity
Describe GPUDirect RDMA and storage paths and the prerequisites (driver, IOMMU, NIC, container permissions) that commonly break them
Your checkpoint
A benchmark report with NCCL all-reduce results across at least three message sizes, an annotated explanation of the knee point, and a ranked list of suspected causes if bandwidth is below expectation.
AI Software Stack and Workload Operations
Compare the layers of the AI software stack (driver, CUDA runtime, cuDNN/cuBLAS, NCCL, container toolkit, orchestration, framework) and state which layer owns which failure class
Describe containerized GPU workload operations: GPU device exposure, MIG-aware scheduling, resource requests/limits, and image reproducibility
Explain job scheduling concerns for AI workloads: gang scheduling, queueing, preemption, and checkpoint/restart policy
Practice version-compatibility reasoning across driver, CUDA, framework, and container image versions
Your checkpoint
A compatibility matrix and troubleshooting decision tree that routes a failing job to the correct stack layer based on its error signature.
AI Software Stack and Workload Operations & AI Infrastructure and Data Center Operations
Instrument an AI workload with utilization, memory, throughput, and power telemetry and distinguish idle-from-bottleneck from genuine saturation
Describe observability and capacity planning practices: trend analysis, headroom targets, and chargeback/showback models for GPU time
Explain multi-tenancy and data governance concerns in an AI factory: isolation, quotas, audit trails, and sensitive-data handling
Analyze a slow-training case end to end, ruling out data loader, host, network, and GPU causes in a defensible order
Your checkpoint
A written performance postmortem for a synthetic slow-training scenario, including the evidence that ruled out each candidate cause and the final root cause.
AI Infrastructure and Data Center Operations & AI Factory Deployment and Management & Accelerated Computing and GPU Operations & AI Networking and Interconnect & AI Software Stack and Workload Operations
Rehearse timed mixed-domain question sets and review every miss down to the underlying concept, not just the answer
Re-derive the highest-risk facts from memory: failure signatures, fabric concepts, stack-layer ownership, and deployment sequencing
Teach two weak areas aloud in plain language to confirm real understanding rather than recognition
Build a final one-page reference sheet of numbers, thresholds, and decision rules you will review on exam day
Your checkpoint
A scored full-length practice run with a written error log, plus a one-page condensed reference sheet covering all five domains.
Milestones
Week 2: You can name five GPU failure signatures and give the correct first triage action for each without notes.
Week 4: You can whiteboard a 64-GPU cluster from bare metal to validated node, including fabric topology and the pass/fail gate at each stage.
Week 6: Given an unfamiliar job error message, you can correctly assign it to a software stack layer and justify the assignment in two sentences.
Week 8: You score at or above your target on a timed mixed-domain practice set and your error log shows no repeated miss in the same domain.