NVIDIA Software Stack and CUDA Ecosystem
Stack layers
From bottom up: GPU driver, CUDA runtime and driver APIs, libraries (cuBLAS, cuDNN, NCCL, TensorRT, DALI), frameworks (PyTorch, TensorFlow, JAX), and application tooling. Questions often ask which layer owns a responsibility. Example: NCCL handles collective communication; cuDNN provides deep-learning primitives; TensorRT builds optimized inference engines.
CUDA programming basics
Know how to compile with nvcc, how host and device memory differ (cudaMalloc, cudaMemcpy, unified memory), and how streams enable asynchronous overlap of compute and transfers. A trap is forgetting that operations in the default stream may serialize; use non-default streams and events for concurrency. Another trap is confusing the CUDA toolkit version with the driver version: the driver must be new enough to support the toolkit, and CUDA has minor-version compatibility within a major release.
Containers and portability
The NVIDIA Container Toolkit exposes GPUs to containers; the CUDA forward-compatibility package lets newer toolkits run on older datacenter drivers. Candidates should recognize that container images bundle user-space libraries but rely on the host driver.
Profiling and debugging
Nsight Systems gives a timeline view (CPU/GPU overlap, gaps); Nsight Compute gives per-kernel metrics (occupancy, memory throughput); compute-sanitizer finds memory errors. A common exam trap is choosing the wrong tool: use Systems for a system-wide timeline, Compute for kernel-level bottleneck analysis.
Worked rule of thumb
Before optimizing a kernel, confirm the GPU is actually the bottleneck: if the Nsight Systems timeline shows long gaps between kernels, the problem is host-side or data-loading, not kernel math. Rule: fix the timeline before tuning the kernel.