GATE DA flashcards

31 cards. Read the term and the definition here. The deck at the top of this page is optional.

Bias-variance trade-off

Total expected squared error decomposes into irreducible noise plus squared bias plus variance. Increasing model complexity typically lowers bias and raises variance; regularization moves the balance toward lower variance.

Precision and recall

Precision is TP/(TP+FP), the fraction of predicted positives that are correct. Recall is TP/(TP+FN), the fraction of actual positives found. Their harmonic mean is the F1 score.

ROC AUC

The area under the true-positive-rate versus false-positive-rate curve. It equals the probability that a randomly chosen positive instance receives a higher score than a randomly chosen negative instance.

Stratified k-fold CV

Cross-validation that partitions data into k folds while preserving the class proportions of the full dataset in each fold. It reduces estimate variance for imbalanced classification compared with plain random folds.

Lasso versus Ridge

Both add a penalty to the loss. Lasso uses the L1 norm and can set coefficients exactly to zero, giving sparse feature selection; Ridge uses the L2 norm and shrinks coefficients smoothly without eliminating them.

Gini impurity

For a node with class proportions p_i, Gini = 1 − Σ p_i². It is zero for a pure node and maximized when classes are evenly mixed. Decision trees split to maximize the resulting impurity decrease.

Gradient descent update rule

Parameters move opposite the gradient: θ ← θ − η·∇L(θ), where η is the learning rate. Too large an η causes divergence or oscillation; too small an η makes convergence very slow.

Softmax function

Maps a vector of logits z to probabilities: softmax(z)_i = exp(z_i)/Σ_j exp(z_j). Outputs are positive and sum to one. Usually paired with cross-entropy loss in multiclass classification.

k-Nearest Neighbors

A non-parametric classifier that predicts the majority label, or mean value, among the k closest training points. Small k means low bias and high variance; feature scaling is required.

Naive Bayes classifier

Applies Bayes' rule assuming features are conditionally independent given the class: P(y|x) ∝ P(y)·Π P(x_i|y). Laplace smoothing prevents zero probabilities from unseen feature values.

Support vector machine margin

An SVM finds the hyperplane maximizing the margin 2/‖w‖ subject to y_i(wᵀx_i + b) ≥ 1. The parameter C controls tolerance of margin violations.

Kernel trick

Replaces explicit dot products with a kernel function K(x, x') such as the polynomial or RBF kernel. This lets a linear-margin SVM learn nonlinear boundaries without explicitly mapping data to high dimensions.

k-means objective

k-means minimizes the within-cluster sum of squared distances to cluster centroids. It requires k in advance, is sensitive to initialization, and assumes roughly spherical clusters of similar size.

Principal component analysis

PCA projects data onto orthogonal directions of maximum variance, found as eigenvectors of the covariance matrix. Components are ranked by eigenvalue, and the explained-variance ratio is λ_i divided by the sum of eigenvalues.

Bagging versus boosting

Bagging trains learners in parallel on bootstrap samples and averages them, mainly reducing variance. Boosting trains weak learners sequentially on the errors of predecessors, mainly reducing bias.

R-squared

R² = 1 − SS_res/SS_tot measures the fraction of variance explained by a regression model. A value of one is perfect fit; zero matches the mean predictor; values can be negative.

Cross-validation purpose

Repeatedly splitting data into training and validation folds estimates generalization error and guides model selection. The test set must stay untouched until final evaluation to avoid optimistic bias.

Bayes' theorem

P(A|B) = P(B|A)·P(A)/P(B). The posterior is proportional to likelihood times prior, with the denominator normalizing over all possible hypotheses. It underpins Naive Bayes and Bayesian inference.

Variance of a sum

For independent random variables, Var(X + Y) = Var(X) + Var(Y). More generally, Var(X + Y) includes twice the covariance term, which vanishes under independence.

Central limit theorem

The sum or mean of many independent, identically distributed variables with finite variance tends toward a normal distribution. The standard error of the sample mean is σ/√n.

p-value interpretation

The probability, assuming the null hypothesis is true, of observing a test statistic at least as extreme as the one obtained. It is not the probability that the null hypothesis is true.

Type I and Type II errors

A Type I error rejects a true null hypothesis and occurs with probability α, the significance level. A Type II error fails to reject a false null; power equals one minus its probability β.

Confidence interval for a mean

With known standard deviation, x̄ ± z_{α/2}·σ/√n. A 95% interval uses z ≈ 1.96. The interval is random across samples; it does not give a probability for a fixed parameter.

Matrix rank and invertibility

Rank equals the number of nonzero singular values and the dimension of the column space. A square matrix is invertible exactly when its determinant is nonzero, equivalently when all eigenvalues are nonzero.

Positive definite matrix

A symmetric matrix is positive definite when xᵀAx > 0 for every nonzero vector x, equivalently when all its eigenvalues are positive. Such matrices arise as covariance and Hessian matrices.

Conditional independence

Events A and B are conditionally independent given C when P(A∩B|C) = P(A|C)·P(B|C). Unconditionally they may still be dependent; this assumption drives Naive Bayes.

Mutable default argument

In Python, a default list or dict in a function signature is created once at definition time, so mutations persist across calls. Use None as the default and construct the object inside the function.

SQL WHERE versus HAVING

WHERE filters individual rows before grouping; HAVING filters aggregated groups after GROUP BY. Aggregate conditions such as COUNT(*) > 5 must appear in HAVING, not WHERE.

Pandas loc versus iloc

loc selects rows and columns by label, and slices include the endpoint. iloc selects by integer position with Python-style exclusive endpoints. Mixing them up is a common indexing bug.

IQR outlier rule

With IQR = Q3 − Q1, values below Q1 − 1.5·IQR or above Q3 + 1.5·IQR are flagged as outliers. The rule is distribution-free but assumes a roughly symmetric spread.

Backpropagation

The chain rule applied through a computational graph to compute loss gradients with respect to every parameter. Gradients then drive an optimizer such as SGD or Adam to update weights.