Probability and Moment Generating Functions
CS1 examines probability generating functions (pgfs) and moment generating functions (mgfs) as compact encodings of a distribution. You need to derive moments, identify distributions from a generating function, and handle sums of independent variables.
Core rules
- For a non-negative integer-valued X, G_X(z)=E[z^X]=Σp(x)z^x; G_X(1)=1, E[X]=G_X'(1), and Var(X)=G_X''(1)+G_X'(1)−[G_X'(1)]².
- The mgf M_X(t)=E[e^{tX}] gives E[X]=M_X'(0), E[X²]=M_X''(0); the cgf is K_X(t)=log M_X(t).
- If X and Y are independent, G_{X+Y}(z)=G_X(z)G_Y(z) and M_{X+Y}(t)=M_X(t)M_Y(t).
- Generating functions are unique when they exist: matching an mgf identifies the distribution.
Worked example
For X~Poisson(λ), G(z)=Σ e^{−λ}λ^x z^x/x! = e^{λ(z−1)}. Then G'(z)=λe^{λ(z−1)}, so G'(1)=λ. Also G''(z)=λ²e^{λ(z−1)}, so G''(1)=λ². Hence Var(X)=λ²+λ−λ²=λ. The same result follows from M(t)=exp(λ(e^t−1)) with M'(0)=λ and M''(0)=λ²+λ.
Traps
- An mgf need not exist for heavy-tailed variables; a pgf applies only to non-negative integer counts.
- Differentiating at the wrong point: pgf moments use z=1, mgf moments use t=0.
- Forgetting that Var(X)=G''(1)+G'(1)−μ², not G''(1)−μ².
- Assuming independence when adding variables; without it the product rule fails.