Estimators and Estimation
This domain asks how estimators are constructed from data and how they are judged - the criteria, the methods, and the asymptotic results that make large-sample inference possible.
Judging an estimator
- Bias: E[theta-hat] - theta. Variance and bias combine in MSE = Var + Bias^2.
- Efficiency: minimum variance within a class, benchmarked by the Cramer-Rao lower bound 1/I(theta), where I(theta) is the Fisher information.
- Consistency: convergence in probability to the true parameter as n grows.
- Sufficiency: a statistic is sufficient if the likelihood factorises into a part depending on the data only through the statistic and a part free of theta.
Methods of estimation
- Method of moments: equate sample and theoretical moments; simple but often inefficient.
- Maximum likelihood: maximise the log-likelihood. Invariance holds, so if theta-hat is the MLE of theta then g(theta-hat) is the MLE of g(theta).
- Asymptotically theta-hat ~ N(theta, 1/I(theta)); the delta method converts this into standard errors for functions of theta-hat.
- Bayesian estimation combines a prior with the likelihood to give a posterior; the posterior mean under a squared-error loss is the Bayes estimator.
- Actuarial link: q-hat_x = d_x/E_x is the binomial MLE, and the Kaplan-Meier estimator is the nonparametric MLE of the survival function.
Worked example
Five exponential claim amounts are 2, 3, 5, 8 and 12, with mean 6. The exponential MLE is lambda-hat = 1/mean = 0.1667. Fisher information per observation is 1/lambda^2, so with n = 5 the standard error is lambda-hat/sqrt(5) = 0.1667/2.236 = 0.0745. Note the parameter estimate is plugged in to compute the standard error, so the reported precision is itself an estimate.
Traps that cost marks
- Confusing the standard deviation of the data with the standard error of the estimate.
- Assuming unbiased always means best; a slightly biased estimator can have lower MSE.
- Using n rather than n-1 in the sample variance when the mean is estimated.