Core Machine Learning and AI Knowledge
What this domain tests
The NCA-GENM core ML domain checks whether you can reason about how generative models are trained and evaluated, not merely name architectures. Expect questions on loss functions, optimization, overfitting, transformer attention, embeddings, and the denoising objective used by diffusion models.
Core rules and concepts
- Discriminative models learn p(y|x); generative models learn p(x) or p(x,y). Classification accuracy is not a proxy for generation quality.
- Cross-entropy for a softmax output is L = -log(p_true). For token models, perplexity = exp(L).
- Attention computes softmax(QK^T / sqrt(d_k))V. The sqrt(d_k) scaling keeps logits from saturating as head dimension grows.
- Overfitting appears as a widening gap between training and validation loss; weight decay, dropout, early stopping, and more data are standard remedies.
- Diffusion models learn to predict the noise added at timestep t, then sample by iteratively denoising from pure noise.
Worked example
A 3-class classifier outputs logits [2.0, 1.0, 0.0] and the true class is 0. The softmax denominator is e^2 + e^1 + e^0 = 7.389 + 2.718 + 1 = 11.107, so p_0 = 7.389 / 11.107 = 0.665 and the loss is -ln(0.665) = 0.408. If the true class were 2 instead, p_2 = 0.090 and the loss would be 2.41. The much larger penalty for a confident wrong answer is exactly what cross-entropy is designed to produce.
Traps that cost marks
- Confusing parameters (weights) with hyperparameters (learning rate, batch size).
- Assuming a lower training loss means a better model.
- Forgetting that softmax is shift-invariant: adding a constant to every logit changes nothing.
- Mixing up precision and recall with loss values.
- Treating a generative model's sample quality as if it were a classification metric.