Bayesian statistics and computational methods
CS2 tests Bayesian inference from first principles and the computational methods needed when conjugate posteriors are unavailable.
Inference
The posterior is p(θ|y)∝p(θ)p(y|θ). Conjugate pairs give closed forms: beta-binomial, gamma-Poisson, normal-normal. Summarise with posterior mean, median, mode and credible intervals. A 95% credible interval contains θ with posterior probability 0.95, given the model and prior. The posterior predictive distribution is p(y_new|y)=∫p(y_new|θ)p(θ|y)dθ.
Computation
Markov chain Monte Carlo draws dependent samples from the posterior. Metropolis–Hastings accepts a proposal θ* with probability min(1, [p(θ*|y)q(θ|θ*)]/[p(θ|y)q(θ*|θ)]). Gibbs sampling updates each parameter from its full conditional. Check convergence with trace plots, multiple chains, R-hat, effective sample size and autocorrelation; discard burn-in and consider thinning. DIC and WAIC compare models, with caveats.
Worked example
Let p have prior Beta(2,3) and observe 7 successes in 10 binomial trials. The posterior is Beta(2+7,3+3)=Beta(9,6). The posterior mean is 9/15=0.60, compared with the prior mean 2/5=0.40 and the MLE 0.70. A 95% equal-tailed credible interval comes from the Beta(9,6) quantiles.
Traps
A credible interval is not a confidence interval. An improper prior can produce an improper posterior. Burn-in does not guarantee convergence. High autocorrelation reduces effective sample size. DIC is not valid for comparing models fitted to different data. Check prior sensitivity.