GATE ST cheat sheet

Indicator of a set A: I_A(x)=1 if x∈A, 0 otherwise; E[I_A]=P(A) and Var(I_A)=P(A)(1−P(A)).

Total probability: P(B)=Σ_i P(B|A_i)P(A_i) for a partition {A_i}; Bayes then gives P(A_j|B)=P(B|A_j)P(A_j)/Σ_i P(B|A_i)P(A_i).

Binomial(n,p): mean np, variance np(1−p); Poisson(λ): mean = variance = λ, and Bin(n,p)→Poisson(np) when n→∞, p→0, np→λ.

Geometric(p) on {1,2,…}: P(X=k)=(1−p)^(k−1)p, mean 1/p, variance (1−p)/p²; memoryless on the integers.

Exponential(λ): f(x)=λe^(−λx), mean 1/λ, variance 1/λ²; the only continuous memoryless law: P(X>s+t|X>s)=P(X>t).

Normal: if XN(μ,σ²) then aX+bN(aμ+b,a²σ²); sums of independent normals are normal; a linear transform of a normal vector stays normal.

Uniform(a,b): mean (a+b)/2, variance (b−a)²/12; Gamma(α,λ) has mean α/λ and variance α/λ², with Exponential(λ)=Gamma(1,λ).

CDF properties: nondecreasing, right-continuous, limits 0 at −∞ and 1 at +∞; P(X=x)=F(x)−F(x⁻) is the jump size.

Cauchy has no mean: E|X|=∞, so the law of large numbers fails; do not apply moment-based results to heavy-tailed laws without checking finiteness.

E[g(X)]=Σ_x g(x)p(x) or ∫g(x)f(x)dx; for a transformed density use f_Y(y)=f_X(g⁻¹(y))·|d/dy g⁻¹(y)| when g is monotone.

Variance identity: Var(X)=E[X²]−(E[X])²; Var(aX+b)=a²Var(X); Var(X+Y)=Var(X)+Var(Y)+2Cov(X,Y).

Covariance is bilinear; Cov(X,Y)=E[XY]−E[X]E[Y]; independence implies zero covariance, but zero covariance does not imply independence.

Correlation ρ=Cov(X,Y)/(σ_Xσ_Y) satisfies |ρ|≤1, with equality iff Y=aX+b almost surely.

MGF: M_X(t)=E[e^(tX)]; if finite on an open interval around 0, it determines the distribution and M_X^(k)(0)=E[X^k].

Independence: f_{X,Y}=f_X f_Y (or p_{X,Y}=p_X p_Y) and F_{X,Y}=F_X F_Y; joint support being a product set plus factorization suffices.

Conditional expectation: E[X]=E[E[X|Y]] (tower property); E[X|Y] is the least-squares-best predictor of X given Y, minimizing E[(X−g(Y))²].

PGF G_X(s)=E[s^X] gives P(X=k)=G_X^(k)(0)/k! and G_X(1)=1; for independent sums G_{X+Y}=G_X G_Y.

Jensen: for convex g, E[g(X)]≥g(E[X]); equality iff X is constant a.s. (strictly convex g).

If Z_1,…,Z_k are iid N(0,1), then ΣZ_i²χ²_k; mean k, variance 2k; if Xχ²_m, Yχ²_n independent, X+Yχ²_{m+n}.

t_ν = Z/√(V/ν) with ZN(0,1), Vχ²_ν independent; as ν→∞, t_ν→N(0,1); t_ν²=F_{1,ν}.

F_{m,n}=(χ²_m/m)/(χ²_n/n) for independent chi-squares; 1/F_{m,n}~F_{n,m}; F quantiles relate to t via F_{1,ν}=t_ν².

For a random sample of size n from N(μ,σ²): X̄N(μ,σ²/n), (n−1)S²/σ²χ²_{n−1}, and X̄ and S² are independent.

Pivot for the mean with unknown σ: (X̄−μ)/(S/√n)~t_{n−1}; with known σ use Z=(X̄−μ)/(σ/√n).

Two-sample: (X̄−Ȳ−(μ₁−μ₂))/(S_p√(1/n₁+1/n₂))~t_{n₁+n₂−2} under equal variances, with S_p²=((n₁−1)S₁²+(n₂−1)S₂²)/(n₁+n₂−2).

Ratio of sample variances: (S₁²/σ₁²)/(S₂²/σ₂²)~F_{n₁−1,n₂−1}; the F test for equal variances is sensitive to non-normality.

Multinomial(n,p₁,…,p_k): counts have means np_i, variances np_i(1−p_i), covariances −np_ip_j; Pearson's Σ(O−E)²/E→χ²_{k−1} asymptotically.

Bivariate normal: conditional X|Y=y is N(μ_X+ρ(σ_X/σ_Y)(y−μ_Y), σ_X²(1−ρ²)); the regression of X on Y is linear.

Estimator T is unbiased for θ if E[T]=θ; the MSE decomposition is MSE(T)=Var(T)+Bias(T)².

Cramér–Rao lower bound: Var(T)≥1/(nI(θ)) for unbiased T, where I(θ)=E[(∂/∂θ log f(X;θ))²]=−E[∂²/∂θ² log f].

An estimator attains the CRLB iff the score factorizes as ∂/∂θ log L(θ)=a(θ)(T−θ); then T is the UMVUE when a complete sufficient statistic exists.

Fisher–Neyman factorization: T is sufficient for θ iff f(x;θ)=g(T(x);θ)h(x); if T is complete and sufficient, any function of T is the unique UMVUE of its mean.

Lehmann–Scheffé: if T is complete sufficient and E[g(T)]=θ, then g(T) is the UMVUE of θ.

Rao–Blackwell: for unbiased T and sufficient S, T*=E[T|S] is unbiased with Var(T*)≤Var(T).

MLE is invariant under reparameterization: if θ̂ maximizes L, then g(θ̂) maximizes the likelihood of g(θ); under regularity, √n(θ̂−θ)→N(0,I(θ)⁻¹).

Method of moments matches E[X^k] to the sample moments; it is consistent under moment conditions but generally not efficient.

Consistency: T_n→θ in probability; sufficient conditions include E[T_n]→θ and Var(T_n)→0 (Chebyshev).

Pivot construction: find Q(θ,X) whose distribution is free of θ, then invert P(a≤Q≤b)=1−α to get the interval.

Known-σ mean interval: X̄±z_{1−α/2}σ/√n; unknown-σ interval: X̄±t_{n−1,1−α/2}S/√n.

Variance interval from a normal sample: [(n−1)S²/χ²_{n−1,1−α/2}, (n−1)S²/χ²_{n−1,α/2}], since chi-square quantiles are not symmetric.

Proportion interval (Wald): p̂±z√(p̂(1−p̂)/n); use Wilson or Clopper–Pearson when np̂ or n(1−p̂) is small.

Neyman–Pearson lemma: the most powerful level-α test of simple H₀ vs simple H₁ rejects for large likelihood ratio L(θ₁)/L(θ₀), with the critical value set by size α.

p-value: the probability, under H₀, of a test statistic at least as extreme as observed; reject H₀ at level α iff p≤α.

Duality: a level-α test corresponds to a 100(1−α)% confidence set, and θ₀ lies outside the interval iff H₀:θ=θ₀ is rejected at level α.

Power = 1−β = P(reject H₀|H₁ true); for a z-test it increases with |μ₁−μ₀|, n, and α, and decreases with σ.

Monotone likelihood ratio in a sufficient statistic T implies a UMP test of one-sided hypotheses of the form T≥c; two-sided alternatives generally lack a UMP test.

Simple linear regression: β̂₁=S_xy/S_xx with S_xy=Σ(x_i−x̄)(y_i−ȳ), S_xx=Σ(x_i−x̄)²; β̂₀=ȳ−β̂₁x̄; the line passes through (x̄,ȳ).

Gauss–Markov: under zero-mean, homoscedastic, uncorrelated errors, OLS is BLUE; normality is needed only for exact t and F tests.

Multiple regression: β̂=(XᵀX)⁻¹Xᵀy, ŷ=Hy with hat matrix H=X(XᵀX)⁻¹Xᵀ; H is symmetric idempotent with trace p (number of parameters).

Residual sum of squares SSE=yᵀ(I−H)y with n−p degrees of freedom; σ̂²=SSE/(n−p) is unbiased when the model is correct.

ANOVA decomposition: SST=SSR+SSE, and under H₀:β₁=…=β_{p−1}=0, (SSR/(p−1))/(SSE/(n−p))~F_{p−1,n−p}.

R²=SSR/SST=1−SSE/SST; adding a regressor never decreases R², so compare models by adjusted R²=1−(1−R²)(n−1)/(n−p).

Coefficient test: t=(β̂j−β_j)/SE(β̂_j)~t{n−p}; SE(β̂j)=σ̂√[(XᵀX)⁻¹]{jj}; multicollinearity inflates these standard errors.

Prediction interval at x₀ is wider than the mean-response interval: Var(ŷ₀−y₀)=σ²(1+h₀₀) versus σ²h₀₀, with h₀₀=x₀ᵀ(XᵀX)⁻¹x₀.

Leverage h_ii∈[0,1] with Σh_ii=p; a point with h_ii near 1 or a large standardized residual (|r_i|>2–3) is influential and should be examined.

Empirical CDF F̂_n(t)=(1/n)ΣI(X_i≤t) is unbiased for F(t) with variance F(t)(1−F(t))/n; by Donsker, √n(F̂_n−F) converges to a Brownian bridge.

Kolmogorov–Smirnov statistic D_n=sup_t|F̂_n(t)−F₀(t)|; its null distribution is distribution-free for continuous F₀, and it detects general differences, not ordered alternatives.

Wilcoxon signed-rank test applies to paired or one-sample data under symmetry about the median; it uses ranks of |X_i| and is distribution-free under H₀: median = 0.

Wilcoxon rank-sum (Mann–Whitney U) tests equality of two distributions under independence; U counts pairs with X_i>Y_j, and the normal approximation uses variance n₁n₂(n₁+n₂+1)/12 under H₀.

Kruskal–Wallis is the rank-based analogue of one-way ANOVA; under H₀ its statistic is approximately χ²_{k−1}, with ties requiring a correction.

Spearman's ρ is Pearson correlation of ranks; under H₀:ρ=0, √(n−1)ρ is approximately N(0,1) for moderate n.

Sign test uses the binomial distribution of the number of positive differences under H₀: median difference = 0; it needs no symmetry assumption but has low power.

Bootstrap: resample with replacement from the data; the nonparametric bootstrap estimates the sampling distribution of a statistic, and the percentile interval uses its α/2 and 1−α/2 quantiles.

Permutation test: pool the data and reassign group labels; the null distribution comes from all relabelings, valid whenever the null is exchangeability of labels.

Bayes: posterior ∝ likelihood × prior, π(θ|x)∝f(x|θ)π(θ); the normalizing constant is the marginal m(x)=∫f(x|θ)π(θ)dθ.

Conjugate pairs: Beta–Binomial, Gamma–Poisson, Normal–Normal (known variance), Dirichlet–Multinomial, and inverse-gamma or half-normal families for variances.

Beta(a,b) prior with Binomial(n,x) data gives Beta(a+x, b+n−x) posterior; posterior mean = (a+x)/(a+b+n) is a weighted average of prior mean and x/n.

Gamma(α,β) prior with Poisson data summing to S over n observations gives Gamma(α+S, β+n) posterior; the posterior mean is (α+S)/(β+n).

Normal prior N(μ₀,τ²) with known σ² and n observations gives posterior mean (μ₀/τ²+n x̄/σ²)/(1/τ²+n/σ²), a precision-weighted average.

Bayes estimator under squared-error loss is the posterior mean; under absolute-error loss it is the posterior median; under 0–1 loss it is the posterior mode.

Credible interval: a set C with P(θ∈C|x)=1−α; unlike a confidence interval, it has a direct posterior probability interpretation given the prior and model.

Bayes risk = E[prior-averaged loss]; the Bayes rule minimizes posterior expected loss pointwise, and the minimax rule minimizes the maximum frequentist risk.

Poisson process rate λ: N(t)~Poisson(λt), independent increments, and interarrival times are iid Exponential(λ); given N(t)=n, arrival times are distributed as order statistics of n uniforms on [0,t].

Poisson merging: independent Poisson processes with rates λ₁,λ₂ merge to rate λ₁+λ₂; splitting each event independently with probability p yields independent Poisson processes of rates λp and λ(1−p).

Random walk S_n=ΣX_i with iid increments: E[S_n]=nμ, Var(S_n)=nσ²; it is recurrent in one and two dimensions and transient in three or more.

Simple symmetric random walk: P(S_{2n}=0)=C(2n,n)/2^{2n}~1/√(πn), so return probabilities decay but their sum diverges.

Branching process with offspring mean m: extinction probability q is the smallest root of s=G(s) in [0,1]; q=1 if m≤1, and q<1 if m>1.

Discrete-time Markov chain: P(X_{n+1}=j|X_n=i, past)=p_ij; the n-step matrix is P^n, and a stationary distribution satisfies π=πP with π≥0, Σπ_j=1.

A finite irreducible aperiodic chain converges to its unique stationary distribution; periodicity prevents convergence but not the existence of π.

Continuous-time Markov chain: generator Q with q_ij for i≠j and q_ii=−Σ_{j≠i}q_ij; stationary π satisfies πQ=0, and holding times in state i are Exponential(−q_ii).

Brownian motion: B(0)=0, independent stationary Gaussian increments with B(t)−B(s)~N(0,t−s), continuous paths; E[B(t)B(s)]=min(s,t).