A fictional policy evaluation with changing groups
A fictional municipality introduces a subsidized transport pass. The following year, average journey costs among surveyed residents fall, and the mayor announces that the pass has solved unequal access. The survey team points out that the second survey includes more students and fewer rural residents than the first. Fuel prices also changed during the period, and a new rail station opened in one district. The evaluation contains expenditure averages but no data on disabled access, service reliability or households that did not obtain a pass. A committee asks students to distinguish the policy objective, financial instrument and possible mechanisms. All events and figures are supplied exercise assumptions rather than a description of a real government program. The task is to design a more credible evaluation and explain how limited evidence should affect the strength of a public claim.
1. What can and cannot be inferred?
Read the worked answer
The surveys show lower reported costs in the observed samples. Changing composition and concurrent events prevent treating the difference alone as the pass’s causal effect or proof that every group now has equal access.
2. Recommend an evaluation approach.
Read the worked answer
Use consistent populations and measures, examine nonparticipants and affected groups, compare plausible alternatives and analyze costs alongside access and reliability. State remaining uncertainty rather than reporting the objective as an achieved result.