I85PREP
⌘ K
NVIDIA NCA-GENM study materials
NVIDIA NCA-GENM study resources

NVIDIA NCA-GENM practice questions

One question at a time. Build your NVIDIA NCA-GENM understanding.

First 50 of 200 questions. Each one lists the choices, the correct answer, and an explanation. The checker at the top of this page saves progress in this browser when it is available.

Continue in the practice tool above to load more questions, 50 at a time.

Question 1Core Machine Learning and AI Knowledge

A generative model reaches 99% accuracy on its training split but only 62% on a held-out validation split. Which condition does this gap most directly indicate?

  1. Vanishing gradients, where early layers receive near-zero updates during backpropagation
  2. Data leakage, where validation examples were accidentally included in the training set
  3. Overfitting, where the model memorizes training-specific patterns that do not generalize Correct answer.
  4. Underfitting, where the model has insufficient capacity to learn the training distribution

Explanation

A large gap between high training performance and much lower validation performance is the classic signature of overfitting: the model fits idiosyncratic noise in the training set rather than generalizable structure. Remedies include stronger regularization, data augmentation, early stopping, and more diverse training data.

Question 2Core Machine Learning and AI Knowledge

What does the softmax function compute from a vector of raw model logits in a generative classifier?

  1. A rank-ordered list of logits sorted from highest to lowest confidence
  2. A binary mask that keeps only the single largest logit and zeroes the rest
  3. A running average of logits accumulated across the current training batch
  4. A normalized probability distribution whose entries are non-negative and sum to one Correct answer.

Explanation

Softmax exponentiates each logit and divides by the sum of exponentials, yielding a categorical probability distribution over classes or vocabulary tokens. This normalization makes outputs interpretable as probabilities and compatible with cross-entropy loss.

Question 3Core Machine Learning and AI Knowledge

In training a neural generative model with stochastic gradient descent, how is each parameter updated relative to the computed gradient?

  1. By setting it to the average of its values across all training examples
  2. By a random value independent of the gradient, to escape local minima
  3. In the opposite direction of the gradient, scaled by a learning rate Correct answer.
  4. In the same direction as the gradient, to increase the loss surface curvature

Explanation

Gradient descent moves parameters opposite the gradient because the gradient points toward steepest increase of the loss. The step size is controlled by the learning rate, and stochastic variants estimate the gradient from mini-batches.

Question 4Core Machine Learning and AI Knowledge

Why does scaled dot-product attention divide the query-key dot product by the square root of the key dimension before applying softmax?

  1. To reduce the number of attention heads needed for a given sequence length
  2. To make the attention weights sum to the key dimension instead of one
  3. To prevent large dot products from saturating softmax and producing tiny gradients Correct answer.
  4. To guarantee that attention weights are exactly uniform across all positions

Explanation

Dot products between high-dimensional query and key vectors grow in magnitude with the dimension, which pushes softmax into saturated regions where gradients nearly vanish. Dividing by the square root of the key dimension keeps the logits at a stable scale and preserves useful gradient flow.

Question 5Core Machine Learning and AI Knowledge

Which objective best describes how a vision-language model such as CLIP is trained on image-text pairs?

  1. Train a separate classifier for every noun that appears in the caption corpus
  2. Maximize cosine similarity for matched pairs while minimizing it for unmatched pairs within a batch Correct answer.
  3. Minimize pixel reconstruction error between each image and its paired caption embedding
  4. Align image and text by predicting the next token in the caption from image patches

Explanation

Contrastive training pulls each image embedding toward its paired caption embedding and pushes it away from all other captions in the batch, and symmetrically for text. This dual-encoder objective yields a shared embedding space usable for zero-shot retrieval and classification.

Question 6Core Machine Learning and AI Knowledge

In a denoising diffusion probabilistic model, what happens during the forward process?

  1. The model directly maps a text prompt to pixels in a single feed-forward pass
  2. A learned network progressively sharpens a blurred image into a clean sample
  3. Gaussian noise is gradually added to the data over many steps until the signal is nearly destroyed Correct answer.
  4. Latent codes are quantized to a discrete codebook and then decoded back to pixels

Explanation

The forward process is a fixed Markov chain that adds small amounts of Gaussian noise at each timestep, eventually approaching a standard normal distribution. Training teaches the reverse process to predict and remove that noise so new samples can be generated from pure noise.

Question 7Core Machine Learning and AI Knowledge

What is the core mechanism of Low-Rank Adaptation (LoRA) when fine-tuning a large generative model?

  1. It freezes the original weights and learns low-rank matrices whose product is added to selected weight updates Correct answer.
  2. It permanently deletes low-magnitude weights to shrink the model before training
  3. It retrains every attention head while freezing the feed-forward layers only
  4. It compresses the training dataset into a smaller synthetic set to reduce epochs

Explanation

LoRA keeps the pretrained weight matrix frozen and injects trainable matrices A and B whose product forms a low-rank update, so far fewer parameters receive gradients. The adapter can later be merged into the base weights or stored separately, which makes fine-tuning memory-efficient.

Question 8Core Machine Learning and AI Knowledge

Why do transformer-based generative models require positional encodings or positional embeddings?

  1. To inject order information, since self-attention is permutation-invariant Correct answer.
  2. To convert token logits into calibrated probabilities at inference
  3. To reduce the vocabulary size before the output projection layer
  4. To allow attention heads to share weights across layers

Explanation

Self-attention compares every token with every other token and produces identical outputs if the input sequence is reordered, so it has no inherent notion of position. Positional encodings or learned embeddings supply that order information, which is essential for language and other sequential modalities.

Question 9Core Machine Learning and AI Knowledge

How does classifier-free guidance influence sampling from a conditional diffusion model?

  1. It trains a separate discriminator to reject samples that ignore the prompt
  2. It increases the number of denoising steps in proportion to the guidance scale
  3. It interpolates between unconditional and conditional score estimates, with a larger scale trading diversity for prompt fidelity Correct answer.
  4. It removes the condition entirely and samples from the unconditional distribution

Explanation

Classifier-free guidance trains the same network with and without conditioning, then extrapolates along the difference between the conditional and unconditional predictions. Increasing the guidance scale strengthens adherence to the prompt but typically reduces sample diversity and can introduce oversaturation artifacts.

Question 10Core Machine Learning and AI Knowledge

Why is layer normalization preferred over batch normalization in transformer blocks for generative language modeling?

  1. It normalizes across the feature dimension per token, stabilizing activations independently of batch size Correct answer.
  2. It normalizes across the batch dimension so that sequence length can vary freely
  3. It replaces the need for residual connections between transformer sublayers
  4. It prevents attention weights from exceeding one by clipping them during training

Explanation

Layer normalization computes statistics over the feature dimension for each token, so its behavior does not depend on how many sequences are in the batch and remains stable with variable-length or small batches. Batch normalization instead depends on batch statistics, which is problematic for autoregressive inference and distributed training.

Question 11Core Machine Learning and AI Knowledge

When training an autoregressive language model on a text corpus, which objective directly matches the usual next-token prediction task?

  1. A hinge loss requiring the correct token logit to exceed all others by one
  2. Mean squared error between predicted token embeddings and the target embedding
  3. Cross-entropy against the target token's one-hot vocabulary distribution Correct answer.
  4. Cosine similarity between hidden states at adjacent sequence positions

Explanation

Next-token prediction treats every position as a multiclass classification over the vocabulary. Cross-entropy maximizes the log-probability assigned to the observed next token, which is the standard training objective. Embedding regression and hinge margins are not the usual formulation for this task.

Question 12Core Machine Learning and AI Knowledge

In text generation, what is the effect of lowering the sampling temperature from 1.0 to 0.5?

  1. It increases the entropy of the output distribution, producing more diverse tokens
  2. It restricts sampling to the single most likely token at every step
  3. It divides logits by a value below 1, sharpening the softmax toward likely tokens Correct answer.
  4. It removes the need for a softmax and samples directly from raw logits

Explanation

Temperature scaling divides each logit by T before the softmax is applied. A temperature below 1 amplifies differences between logits, making the distribution sharper and high-probability tokens more likely. Greedy decoding is a separate argmax procedure, not a temperature setting.

Question 13Core Machine Learning and AI Knowledge

For evaluating a generative image model, what does the Fréchet Inception Distance (FID) compare?

  1. The number of latent modes covered by the generator during sampling
  2. The accuracy of a classifier trained only on generated images
  3. Inception feature distributions of real versus generated images Correct answer.
  4. Pixel-wise mean squared error between generated and real image batches

Explanation

FID extracts features from real and generated images using a pretrained Inception network, fits Gaussian statistics to each feature set, and computes the Fréchet distance between them. Lower FID indicates that generated and real feature distributions are closer. It is not a pixel-level error metric.

Question 14Core Machine Learning and AI Knowledge

During training of a deep generative network, the gradient norm occasionally spikes to very large values. What is the primary purpose of applying gradient norm clipping before the optimizer step?

  1. It scales the whole gradient vector so its norm stays under a threshold Correct answer.
  2. It normalizes every parameter's gradient to unit variance across the batch
  3. It automatically raises the learning rate when gradients become large
  4. It zeroes out individual gradient components that fall below a threshold

Explanation

Global norm clipping rescales the entire gradient vector when its norm exceeds a chosen threshold, which caps the size of the parameter update. This helps prevent destructive updates and divergence caused by rare gradient spikes. It does not adapt the learning rate or modify individual components selectively.

Question 15Core Machine Learning and AI Knowledge

During autoregressive inference with a transformer decoder, what does the key-value (KV) cache primarily avoid recomputing?

  1. The final softmax normalization over the full vocabulary at each step
  2. The key and value projections for tokens already processed in earlier steps Correct answer.
  3. The embedding lookup for the current, newly generated token
  4. The tokenizer vocabulary lookup for previously generated tokens

Explanation

A KV cache stores the key and value tensors for each layer and each past position. When a new token is generated, only its query, key, and value need to be computed, while attention reuses the cached keys and values from the prefix. This reduces repeated projection work during decoding.

Question 16Core Machine Learning and AI Knowledge

Which statement correctly describes BLEU as a metric for generated text?

  1. It computes the edit distance between generated and reference strings at the character level
  2. It combines modified n-gram precision with a brevity penalty against one or more references Correct answer.
  3. It reports the perplexity of a reference language model on the generated sentence
  4. It measures semantic entailment between a generated sentence and a reference using an NLI model

Explanation

BLEU aggregates modified n-gram precision, typically up to four-grams, and multiplies by a brevity penalty to discourage overly short outputs. It compares against one or more reference translations and rewards n-grams that match them. It does not rely on entailment models or character edit distance.

Question 17Core Machine Learning and AI Knowledge

In training a transformer-based generative model, what is the role of dropout applied to hidden activations?

  1. It permanently removes low-magnitude activations to sparsify the network
  2. It quantizes activations to lower precision to reduce memory use
  3. It randomly zeroes activations during training and rescales survivors Correct answer.
  4. It averages multiple independently initialized models at inference time

Explanation

Dropout randomly sets a fraction of activations to zero during training and scales the remaining values to preserve expected magnitude. This discourages units from relying on specific other units and acts as a regularizer. At inference time, dropout is disabled rather than used to average separate models.

Question 18Core Machine Learning and AI Knowledge

In mixed-precision training of a generative model, what is the main reason for maintaining a master copy of the weights in FP32 while computing forward and backward passes in FP16 or BF16?

  1. To retain small weight updates that low-precision rounding would discard Correct answer.
  2. To eliminate the need for loss scaling during backpropagation
  3. To double the effective batch size without increasing memory
  4. To make inference outputs exactly reproducible across different GPUs

Explanation

Low-precision weights can round away tiny gradient updates, especially when the update is small relative to the weight magnitude. Keeping an FP32 master copy allows those updates to accumulate with greater precision before the next low-precision cast. Loss scaling may still be needed, particularly for FP16 training.

Question 19Core Machine Learning and AI Knowledge

In latent diffusion models, why is the diffusion process typically run in the latent space of a pretrained autoencoder rather than directly in pixel space?

  1. It makes the forward noising process deterministic and independent of a schedule
  2. It guarantees that generated image pixels are statistically independent
  3. It reduces denoising cost at lower spatial resolution while the decoder restores detail Correct answer.
  4. It eliminates the need to train or use a separate autoencoder decoder

Explanation

A pretrained autoencoder maps images into a lower-resolution latent space where denoising is substantially cheaper than operating on full pixel grids. The decoder then maps the denoised latent back to a detailed image. The noise schedule and stochastic forward process are still part of diffusion training.

Question 20Core Machine Learning and AI Knowledge

In retrieval-augmented generation (RAG), what is the main purpose of retrieving passages before the language model generates an answer?

  1. To replace the language model's decoder with a nearest-neighbor lookup
  2. To condition generation on external evidence, reducing hallucination Correct answer.
  3. To guarantee that every generated token appears verbatim in a retrieved passage
  4. To fine-tune the model's weights on the retrieved passages during inference

Explanation

RAG retrieves relevant passages from an external corpus and inserts them into the model's context so generation is grounded in that evidence. This can improve factual accuracy and freshness without updating the model's parameters. The model still generates freely and is not forced to copy retrieved text verbatim.

Question 21Core Machine Learning and AI Knowledge

In a transformer-based generative language model, what is the primary function of the token embedding layer?

  1. It maps each discrete token ID to a learned dense vector representation. Correct answer.
  2. It normalizes each token's hidden vector before the attention sublayer.
  3. It computes attention weights between all pairs of token positions.
  4. It caches previously computed keys and values during autoregressive decoding.

Explanation

The token embedding layer is a lookup table that converts integer token IDs into continuous dense vectors, which are then processed by transformer blocks. Positional information is normally added separately, and attention and caching are handled by other components.

Question 22Core Machine Learning and AI Knowledge

In a generative adversarial network, what is the discriminator trained to do?

  1. Estimate the exact likelihood of every training example under an explicit density model.
  2. Distinguish real samples from samples produced by the generator. Correct answer.
  3. Generate realistic samples from random noise vectors.
  4. Encode inputs into a lower-dimensional latent representation.

Explanation

A GAN trains two networks adversarially: the generator tries to produce realistic samples, while the discriminator learns to classify inputs as real or generated. This adversarial signal pushes the generator toward more realistic outputs.

Question 23Core Machine Learning and AI Knowledge

Why do many generative language models use subword tokenization such as byte-pair encoding (BPE) instead of word-level tokenization?

  1. It makes the model's context window effectively unlimited.
  2. It guarantees that every word maps to exactly one token.
  3. It removes the need for an embedding layer in the model.
  4. It balances vocabulary size with coverage of rare and unseen words. Correct answer.

Explanation

Subword tokenization splits text into frequently occurring units, so common words stay short while rare words can be represented as combinations of smaller pieces. This keeps the vocabulary manageable and reduces out-of-vocabulary failures compared with pure word-level tokenization.

Question 24Core Machine Learning and AI Knowledge

In a variational autoencoder, what does the Kullback-Leibler (KL) divergence term in the training objective encourage?

  1. It causes the encoder to ignore the input and emit a constant code.
  2. It forces the decoder to reconstruct inputs with pixel-perfect accuracy.
  3. It keeps the approximate posterior close to the prior distribution. Correct answer.
  4. It encourages each latent dimension to be perfectly independent of the others.

Explanation

The VAE objective combines a reconstruction term with a KL term that regularizes the encoder's approximate posterior toward the prior, usually a standard Gaussian. This regularization shapes the latent space so that sampling from the prior produces meaningful generations.

Question 25Core Machine Learning and AI Knowledge

Compared with greedy decoding, what is the main behavior of beam search when generating text from a language model?

  1. It always produces a shorter sequence than greedy decoding.
  2. It computes exact global probabilities for all possible sequences.
  3. It samples randomly from the full vocabulary at every generation step.
  4. It maintains several candidate partial sequences and selects high-scoring complete sequences. Correct answer.

Explanation

Beam search keeps a fixed number of top-scoring partial hypotheses at each step and expands them, instead of committing to only the single best next token as greedy decoding does. It is still an approximate search method and does not enumerate every possible sequence.

Question 26Core Machine Learning and AI Knowledge

How should a perplexity score be interpreted for a generative language model evaluated on a test corpus?

  1. Higher perplexity always means better sample diversity.
  2. Perplexity measures the wall-clock latency of text generation.
  3. Perplexity is the exact percentage of tokens predicted correctly.
  4. Lower perplexity indicates the model assigns higher probability to the observed tokens. Correct answer.

Explanation

Perplexity is derived from the average negative log-likelihood assigned to the test tokens, so lower values mean the model is less surprised by the data. It is not an accuracy percentage, a diversity measure, or a timing metric.

Question 27Core Machine Learning and AI Knowledge

What is a primary benefit of using multiple attention heads in a transformer generative model?

  1. They reduce the total number of parameters compared with a single head of the same total dimension.
  2. They guarantee that attention weights are sparse and easily interpretable.
  3. They eliminate the need for positional information in the model.
  4. They allow different heads to learn distinct attention patterns in parallel. Correct answer.

Explanation

Multi-head attention projects queries, keys, and values into several subspaces so that each head can focus on different relationships or patterns. The outputs are then combined, giving the model richer representational capacity than a single attention head.

Question 28Core Machine Learning and AI Knowledge

Why is a learning rate warmup schedule commonly used when training transformer-based generative models?

  1. It permanently lowers the learning rate to zero after the first epoch.
  2. It guarantees convergence to the global optimum of the training objective.
  3. It reduces early update magnitudes while optimizer statistics are still stabilizing. Correct answer.
  4. It removes the need for gradient clipping during training.

Explanation

Warmup gradually increases the learning rate from a small value during the first training steps, which helps prevent large unstable updates while adaptive optimizer estimates are still forming. It improves stability but does not guarantee global optimality or replace other safeguards such as gradient clipping.

Question 29Core Machine Learning and AI Knowledge

When a large generative model is quantized from FP16 to INT8 for inference, what is the typical trade-off?

  1. Guaranteed identical outputs to the FP16 model for every input.
  2. Reduced memory footprint and faster inference with some potential accuracy loss. Correct answer.
  3. Elimination of the need for calibration or scaling factors.
  4. Higher numerical precision with increased memory bandwidth usage.

Explanation

INT8 quantization stores weights and often activations in 8-bit integers, which lowers memory use and can speed up inference on suitable hardware. The reduced numerical range can introduce quantization error, so accuracy may degrade unless calibration and careful scaling are used.

Question 30Core Machine Learning and AI Knowledge

During full fine-tuning of a pretrained generative model on a narrow domain, what does catastrophic forgetting refer to?

  1. The attention heads becoming permanently disabled by dropout during training.
  2. The tokenizer failing to encode domain-specific vocabulary.
  3. Degradation of previously learned general capabilities as weights shift toward the new domain. Correct answer.
  4. The training loss suddenly diverging because the learning rate is too high.

Explanation

Catastrophic forgetting occurs when updating all model parameters on a narrow dataset overwrites or degrades knowledge acquired during pretraining. Mitigations include replaying general data, adding regularization, or using parameter-efficient methods that constrain how much the original weights change.

Question 31Core Machine Learning and AI Knowledge

In a decoder-only transformer used for text generation, what does the causal attention mask enforce during training?

  1. Each position can attend only to tokens from other sequences in the batch.
  2. Each position can attend only to the final token in the sequence.
  3. Each position can attend only to itself and earlier positions. Correct answer.
  4. Each position can attend to every position in the sequence.

Explanation

A causal mask sets attention weights for future positions to zero, so each prediction depends only on left context. This matches the next-token prediction objective and prevents the model from seeing the target token before predicting it.

Question 32Core Machine Learning and AI Knowledge

At inference time, what is the primary purpose of classifier-free guidance in a diffusion model?

  1. To force the sampler to finish the entire generation in a single denoising step.
  2. To estimate FID from the model's training loss during sampling.
  3. To train the denoiser without class labels or text prompts.
  4. To strengthen conditioning by mixing conditional and unconditional predictions. Correct answer.

Explanation

Classifier-free guidance combines a conditional prediction with an unconditional one, then extrapolates away from the unconditional direction. A larger guidance scale usually strengthens prompt adherence but can reduce diversity or introduce artifacts.

Question 33Core Machine Learning and AI Knowledge

In text generation, what does top-p (nucleus) sampling do?

  1. It samples from the smallest token set whose cumulative probability exceeds p. Correct answer.
  2. It removes tokens whose individual probability falls below a fixed count p.
  3. It samples uniformly from the full vocabulary without probability weighting.
  4. It always selects the highest-probability token at every step.

Explanation

Top-p sampling sorts tokens by probability and keeps the shortest prefix whose cumulative mass reaches p. It then renormalizes and samples from that nucleus, so the candidate set size adapts to the shape of the distribution.

Question 34Core Machine Learning and AI Knowledge

Why does low-rank adaptation (LoRA) reduce memory during fine-tuning of a large generative transformer?

  1. It recomputes activations during the backward pass instead of storing them.
  2. It deletes half of the transformer layers before the fine-tuning run starts.
  3. It freezes pretrained weights and trains small low-rank matrices in some layers. Correct answer.
  4. It stores the full optimizer state in 8-bit integers for every model parameter.

Explanation

LoRA keeps the original weight matrix frozen and learns a low-rank update, so gradients and optimizer states are needed only for the small adapter matrices. This greatly lowers trainable-parameter memory while preserving the base model.

Question 35Core Machine Learning and AI Knowledge

What does a CLIP-style contrastive objective primarily align during pretraining?

  1. Discriminator scores and generator outputs in an adversarial game.
  2. Image embeddings and matching text embeddings in a shared space. Correct answer.
  3. Token logits and human preference rankings from reinforcement learning.
  4. Generated pixels and ground-truth pixels at each diffusion timestep.

Explanation

CLIP-style training pulls the embeddings of matched image-text pairs together and pushes unmatched pairs apart within a batch. The resulting shared space supports zero-shot classification and text-conditioned generation.

Question 36Core Machine Learning and AI Knowledge

In GAN training for image generation, what is mode collapse?

  1. The discriminator becomes too strong and outputs constant real scores.
  2. The latent vectors are normalized to unit length before each generator update.
  3. The generator produces limited variety, repeatedly outputting similar samples. Correct answer.
  4. The training loss decreases smoothly until the model reaches zero error.

Explanation

Mode collapse occurs when the generator maps many latent inputs to a narrow set of outputs that fool the discriminator. It reflects missing coverage of the data distribution rather than simply a strong discriminator or a low loss value.

Question 37Core Machine Learning and AI Knowledge

What role do positional embeddings play in a transformer-based generative model?

  1. They inject information about token order that attention alone does not encode. Correct answer.
  2. They normalize hidden states to stabilize the residual stream during training.
  3. They compress the vocabulary so rare tokens share identical representations.
  4. They convert the model's output logits into probabilities for sampling.

Explanation

Self-attention is permutation-invariant without position information, so positional embeddings let the model distinguish token order. They are added to or combined with token representations before attention layers.

Question 38Core Machine Learning and AI Knowledge

What is the main trade-off of activation checkpointing during training of a large generative model?

  1. It converts all model weights to INT4, reducing both memory and numerical range.
  2. It reduces activation memory at the cost of extra computation during backpropagation. Correct answer.
  3. It eliminates the need for gradient synchronization across data-parallel workers.
  4. It increases GPU memory usage so that no forward activations need to be recomputed.

Explanation

Activation checkpointing stores only selected activations and recomputes the rest during the backward pass. This lowers peak memory enough to fit larger models or batches, but adds forward computation to training time.

Question 39Core Machine Learning and AI Knowledge

Why is near-duplicate removal important when curating a very large text corpus for generative model pretraining?

  1. Duplicates prevent the tokenizer from building a subword vocabulary.
  2. Duplicates block the use of mixed-precision arithmetic on tensor cores.
  3. Duplicates force the model to enlarge its context window during pretraining.
  4. Duplicates can cause memorization and overrepresent repeated content. Correct answer.

Explanation

Repeated passages increase their effective sampling weight and make verbatim memorization more likely. Deduplication improves distributional balance and reduces wasted training compute on redundant text.

Question 40Core Machine Learning and AI Knowledge

When sampling from a diffusion model, what is the general effect of increasing the number of denoising steps?

  1. It guarantees exact log-likelihood computation for the generated image.
  2. It removes the need for a noise schedule during training.
  3. It can improve sample quality while increasing inference time. Correct answer.
  4. It always makes the generated samples less diverse and lower quality.

Explanation

More denoising steps give a finer discretization of the reverse process, which can improve fidelity but costs more model evaluations. The benefit diminishes, and fast samplers often trade some quality for speed.

Question 41Data Analysis

A team is fine-tuning a multimodal generative model on an image–caption dataset in which some object categories appear far more often than others. They want each of the training, validation, and test splits to preserve the original category proportions. Which splitting strategy should they use?

  1. Place all rare categories in the test split and common categories in training.
  2. Stratified splitting by category label so each split mirrors the overall class distribution. Correct answer.
  3. Split by image resolution so each split contains the same range of pixel dimensions.
  4. Randomly assign files to splits without considering their category labels.

Explanation

Stratified splitting samples within each category so every split reflects the dataset's class proportions, which is important when categories are imbalanced. Random splitting can by chance underrepresent rare categories, and intentionally isolating rare classes in test prevents the model from learning them. Splitting by resolution does not control label distribution.

Question 42Data Analysis

In preparing text captions for a generative model, a data engineer uses byte-pair encoding (BPE). What is the primary benefit of subword tokenization in this context?

  1. It encodes rare or unseen words as sequences of frequent subunits, reducing OOV issues. Correct answer.
  2. It eliminates the need to pad or truncate sequences when forming batches.
  3. It encrypts the captions so that sensitive text cannot be reconstructed.
  4. It guarantees that the number of tokens always equals the number of whitespace-delimited words.

Explanation

BPE learns frequent character and subword units and can decompose unseen words into those units, which limits out-of-vocabulary failures and controls vocabulary size. Token counts generally differ from word counts, padding is still needed for batching, and tokenization is not encryption.

Question 43Data Analysis

Before feeding images into a neural network, a common preprocessing step is to scale pixel values. Which transformation correctly describes standard min–max scaling of 8-bit images?

  1. Divide pixel values by 255 to map 8-bit intensities to [0, 1]. Correct answer.
  2. Apply one-hot encoding to each pixel intensity value.
  3. Resize every image to 224×224 pixels regardless of aspect ratio.
  4. Convert every image to grayscale so that only one channel remains.

Explanation

Min–max scaling maps the 0–255 8-bit pixel range to [0, 1] by dividing each pixel value by 255. Resizing and grayscale conversion are separate preprocessing choices, and one-hot encoding is not used for continuous pixel intensities.

Question 44Data Analysis

When normalizing images for a multimodal model, a team computes the per-channel mean and standard deviation from the full dataset, including the held-out test set, and applies those statistics everywhere. What is the main methodological problem?

  1. Leakage from the test set makes the reported test performance optimistically biased. Correct answer.
  2. Per-channel statistics cannot be used with color images.
  3. The normalization statistics are computationally too expensive to estimate on the full dataset.
  4. Normalization should always use a mean of 0.5 and a standard deviation of 0.5.

Explanation

Using test data to compute normalization statistics introduces data leakage because the preprocessing depends on the evaluation set. The statistics should be estimated from the training split and then applied unchanged to validation and test data. Cost is not the central methodological issue, and per-channel statistics are standard for color images.

Question 45Data Analysis

A text-to-image dataset contains many examples of common objects but very few examples of certain rare objects. Which training-time strategy most directly addresses this class imbalance without discarding data?

  1. Increase the batch size while keeping the sampling distribution unchanged.
  2. Evaluate the model only on the common-object examples.
  3. Use weighted sampling so rare-object examples are drawn more frequently. Correct answer.
  4. Remove all rare-object examples so the classes are balanced.

Explanation

Weighted sampling assigns higher selection probability to underrepresented examples, so rare classes contribute more to each epoch without deleting data. Increasing batch size alone does not change the relative frequency of classes, removing rare examples loses information, and restricting evaluation hides performance on rare objects.

Question 46Data Analysis

After encoding images and text into CLIP embedding vectors, a retrieval system ranks images by relevance to a text query. The embeddings are L2-normalized. Which similarity measure is most appropriate for ranking?

  1. Levenshtein distance between the image filename and the query string.
  2. Euclidean distance computed on the original, unnormalized embeddings.
  3. Jaccard similarity between the raw pixel values.
  4. Cosine similarity, equal to the dot product for L2-normalized vectors. Correct answer.

Explanation

CLIP embeddings are compared by cosine similarity, and after L2 normalization the cosine similarity is the dot product. Cosine similarity measures angle rather than magnitude, which suits embedding retrieval. Jaccard and Levenshtein operate on sets and strings, not dense embeddings.

Question 47Data Analysis

A large image–text corpus contains many near-duplicate images that differ only by slight cropping, compression, or brightness. Which approach is best suited to detecting such near-duplicates at scale?

  1. Flag pairs with perceptual-hash or embedding similarity above a threshold. Correct answer.
  2. Match images by their original filenames.
  3. Group images by their pixel dimensions and treat each group as duplicates.
  4. Compare exact MD5 checksums of the raw image files.

Explanation

Perceptual hashing and embedding-based similarity capture visual content and remain stable under minor transformations, so they can flag near-duplicates that byte-level checks miss. Exact MD5 hashes change with any byte difference, filenames are unreliable, and dimensions alone do not indicate duplicate content.

Question 48Data Analysis

A team evaluates a text-to-image generative model using the Fréchet Inception Distance (FID) between generated and real images and obtains a score of 18.7. How should they interpret this result?

  1. Lower FID means the generated distribution is closer to the real one; compare it with baselines. Correct answer.
  2. FID measures pixel-wise mean squared error between each generated image and its nearest real image.
  3. FID is the percentage of generated images that a human judge classifies as realistic.
  4. Higher FID is better because it indicates greater diversity in generated images.

Explanation

FID compares the distributions of Inception features for real and generated images, and lower values indicate greater similarity to the real distribution. Because FID is not an absolute percentage or an error in pixel space, the score is interpreted by comparing models or runs evaluated the same way.

Question 49Data Analysis

A multimodal dataset uses human annotators to assign one of five scene categories to each image. Before training, the team wants to quantify agreement between annotators while correcting for agreement expected by chance. Which statistic is appropriate?

  1. Cosine similarity, which compares the angle between embedding vectors.
  2. FID, which compares feature distributions of real and generated images.
  3. Cohen's kappa, which corrects observed categorical agreement for chance. Correct answer.
  4. BLEU, which compares n-gram overlap between generated and reference captions.

Explanation

Cohen's kappa is designed for categorical labels and subtracts the agreement expected by chance from observed agreement. BLEU, FID, and cosine similarity address text overlap, image distributions, and embedding similarity, not inter-annotator agreement on category labels.

Question 50Data Analysis

A data scientist applies principal component analysis (PCA) to 512-dimensional CLIP image embeddings to visualize them in two dimensions. Which statement about PCA is correct in this setting?

  1. PCA finds orthogonal linear directions of greatest variance in the data without using labels. Correct answer.
  2. PCA is a nonlinear method that preserves local neighborhood structure better than any linear method.
  3. PCA guarantees that pairwise distances between all points are exactly preserved in the two-dimensional projection.
  4. PCA is a supervised method that maximizes separation between labeled classes.

Explanation

PCA is an unsupervised linear technique that projects data onto orthogonal components ordered by explained variance; it does not use class labels. It does not preserve all pairwise distances exactly, and nonlinear manifold methods such as t-SNE or UMAP are separate techniques.