Sampling Distributions and Estimation
CS1 tests sampling distributions, point estimation and confidence intervals. You must distinguish a statistic from a parameter, derive method-of-moments and maximum-likelihood estimators, and use standard sampling results.
Core rules
- A random sample X_1,...,X_n is iid with the population distribution. The sample mean has E[X̄]=μ and Var(X̄)=σ²/n.
- For normal populations, X̄
N(μ,σ²/n), (n−1)S²/σ²χ²_{n−1}, and the t statistic has a t_{n−1} distribution when σ is unknown. - An estimator is unbiased if E[θ̂]=θ; MSE(θ̂)=Var(θ̂)+Bias².
- The likelihood is L(θ)=Πf(x_i;θ). Maximise log L; check the second derivative or boundary. The MLE is invariant under transformations.
Worked example
Let X_1,...,X_n be iid Exponential(λ) with density λe^{−λx}. Then L(λ)=λ^n exp(−λΣx_i) and ℓ=n log λ−λΣx_i. Setting ℓ'(λ)=n/λ−Σx_i=0 gives λ̂=1/X̄. Since ℓ''(λ)=−n/λ²<0, this is a maximum. Note E[λ̂] is not λ in general; unbiasedness of 1/X̄ is not automatic.
Traps
- Confusing the sample standard deviation s (denominator n−1) with the population σ.
- Reporting an MLE without checking it lies in the parameter space.
- Assuming unbiasedness is preserved by nonlinear transformations.
- Using normal critical values when σ is estimated from a small sample; use t.