Bayesian Statistics
CS1 tests Bayesian inference: prior, likelihood and posterior, conjugate updating, and the interpretation of Bayesian intervals and predictions.
Core rules
- Posterior ∝ prior × likelihood: p(θ|x) ∝ p(θ)L(θ).
- Conjugate pairs include Beta prior with binomial likelihood, Gamma prior with Poisson likelihood, and Normal prior with Normal likelihood (known variance).
- The posterior mean is a Bayes estimator under squared error loss; the posterior median under absolute error.
- A 95% credible interval has 95% posterior probability of containing θ, given the data.
- The posterior predictive distribution integrates the sampling distribution over the posterior.
Worked example
Let θ be a success probability with prior θBeta(a,b), and observe x successes in n Bernoulli trials. The likelihood is proportional to θ^x(1−θ)^{n−x}. Multiplying gives posterior kernel θ^{a+x−1}(1−θ)^{b+n−x−1}, so θ|xBeta(a+x,b+n−x). The posterior mean is (a+x)/(a+b+n), a weighted average of the prior mean a/(a+b) and the sample proportion x/n. As n grows, the data dominate. For a=2,b=2,x=7,n=10, the posterior is Beta(9,5) with mean 9/14≈0.643.
Traps
- Saying a credible interval means 95% of repeated samples capture θ; that is a confidence interval.
- Forgetting the normalising constant is unnecessary only when the posterior is recognisable.
- Using an improper prior without checking the posterior is proper.
- Double-counting the data in the prior or predictive distribution.