GATE DA key terms

The balance between a model's tendency to miss real patterns (bias) and its sensitivity to random noise in training data (variance). Reducing one usually increases the other, so tuning aims for the lowest total error.

A model fits training data so closely, including its noise, that it generalizes poorly to new samples. Detected when training error stays low while validation error rises.

Adding a penalty on model complexity to the training objective so weights stay small and generalization improves. L1 (lasso) encourages sparsity; L2 (ridge) shrinks weights smoothly.

A resampling method that splits data into folds, trains on some and validates on the rest, then averages results. It estimates generalization error without wasting data on a single fixed holdout.

A setting chosen before training rather than learned from data, such as learning rate, tree depth, or regularization strength. It is typically tuned by search over a validation scheme.

A principle that picks parameter values maximizing the probability of the observed data under an assumed model. For Gaussian noise it reduces to least squares; it underlies many estimators.

A rule relating conditional probabilities: the posterior equals the likelihood times the prior divided by the evidence. It converts prior beliefs into updated beliefs after observing data.

A probabilistic classifier applying Bayes' theorem while assuming features are conditionally independent given the class. Despite the strong assumption, it is fast and often competitive for text data.

A linear classifier that maps a weighted feature sum through the sigmoid function to output class probabilities. Trained by minimizing cross-entropy loss; decision boundary is linear in feature space.

A smooth S-shaped function mapping any real number into the interval between zero and one. Common in binary classification outputs and historically in neural network activations.

A function converting a vector of real scores into a probability distribution over classes by exponentiating and normalizing. Used as the output layer for multi-class classification.

A loss measuring disagreement between predicted class probabilities and true labels, equal to negative log-likelihood of the correct class. It penalizes confident wrong predictions heavily.

An iterative optimization method that repeatedly steps parameters opposite the gradient of the loss to reduce it. Step size is controlled by the learning rate; variants use batches of data.

An algorithm computing gradients of a loss with respect to every network parameter using the chain rule, propagating errors backward through layers. It makes training deep networks computationally feasible.

A neural architecture using shared-weight filters slid over spatial inputs to detect local patterns, followed by pooling and dense layers. Weight sharing gives translation tolerance and fewer parameters than fully connected nets.

A downsampling operation, such as max or average over small windows, that shrinks spatial dimensions and adds slight invariance to small shifts. It has no learnable weights.

A classifier seeking the separating hyperplane with the largest margin between classes. Slack variables allow violations, and kernel functions extend it to nonlinear boundaries.

Computing inner products in a high-dimensional feature space directly via a kernel function on original inputs, avoiding explicit transformation. Enables linear methods to fit nonlinear boundaries cheaply.

The distance between a decision boundary and the nearest training points of either class. Maximizing it tends to improve generalization; those nearest points are the support vectors.

A dimensionality-reduction technique projecting data onto orthogonal directions of maximum variance. Components are eigenvectors of the covariance matrix; it is unsupervised and lossy.

A factorization of any matrix into two orthogonal matrices and a diagonal matrix of nonnegative singular values. It underlies PCA, low-rank approximation, and pseudoinverse computation.

A scalar such that multiplying a matrix by some nonzero vector yields that scalar times the vector. It measures how much the matrix stretches along that eigenvector direction.

The phenomenon that in high-dimensional spaces data become sparse, distances concentrate, and required sample sizes grow explosively. It motivates dimensionality reduction and careful feature selection.

An unsupervised algorithm partitioning points into k groups by alternating assignment to the nearest centroid and recomputation of centroids. It minimizes within-cluster squared distance and needs k in advance.

A clustering approach building nested groups either by merging closest clusters or splitting, producing a dendrogram. The number of clusters can be chosen after inspecting the tree.

A cluster-quality measure comparing each point's average distance to its own cluster versus the nearest other cluster. Values near one indicate well-separated compact clusters; near zero, overlap.

Precision is the fraction of predicted positives that are correct; recall is the fraction of actual positives found. They trade off, so F1, their harmonic mean, summarizes both.

A plot of true positive rate against false positive rate across all classification thresholds. The area under it summarizes ranking quality, insensitive to class balance.

A measure of uncertainty or average surprise of a random variable, maximized by uniform distributions. In decision trees it quantifies impurity when choosing splits.

The reduction in entropy of a target variable after splitting on an attribute. Decision-tree algorithms such as ID3 select the split with the largest gain.

A nonnegative, asymmetric measure of how one probability distribution differs from another, equal to expected log ratio. It is not a true distance because it violates symmetry and triangle inequality.

A stochastic process where the next state depends only on the current state, not the full history. Described by a transition matrix whose rows sum to one.

A probability distribution over states that remains unchanged after applying the transition matrix. For an irreducible aperiodic chain, it is the long-run fraction of time spent in each state.

A procedure comparing observed data against a null hypothesis using a test statistic and its sampling distribution. A small p-value indicates data unlikely under the null.

The probability, assuming the null hypothesis is true, of obtaining a test statistic at least as extreme as observed. It is not the probability that the null is true.

A range computed from data that would contain the true parameter in a stated fraction of repeated samples. Its width shrinks with larger samples or lower confidence level.

A property of symmetric matrices whose quadratic form is positive for every nonzero vector, equivalent to all eigenvalues being positive. Guarantees strict convexity and unique minima.