Estimation Theory
Estimation Theory tests point and interval estimation: which statistic estimates which parameter, what makes an estimator good, and how to attach a confidence statement.
Properties of estimators
An estimator T of θ is unbiased if E(T) = θ, consistent if it converges to θ in probability as n → ∞, efficient if it has the smallest variance among unbiased estimators, and sufficient if it captures all the information in the sample about θ. x̄ is unbiased for μ; s² = Σ(x − x̄)²/(n − 1) is unbiased for σ², whereas dividing by n gives a biased estimator.
Methods
Maximum likelihood maximises the likelihood, or equivalently the log-likelihood. The method of moments equates sample moments to population moments and solves for the parameter.
Interval estimation
With known σ: x̄ ± z_(α/2)·σ/√n, where z = 1.96 at 95% and 2.58 at 99%. With unknown σ and small n, replace z by t with n − 1 degrees of freedom. For a proportion, p̂ ± z√(p̂q̂/n). The sample size for a margin of error E is n = (zσ/E)², rounded up.
Worked example
A sample of 25 gives x̄ = 52, and σ = 5 is known. The 95% interval is 52 ± 1.96(5/5) = 52 ± 1.96, that is (50.04, 53.96).
Traps
Dividing by n instead of n − 1 for s²; using z when σ is unknown and n is small; saying the probability is 0.95 that μ lies in a computed interval; forgetting to round the sample size upwards.