Exam Scores and a Suspicious Correlation
A university reviews admission data for 1,200 applicants. Variables include secondary-school grade point average (GPA), a standardised entrance test score, whether the applicant attended a fee-paying school, and first-year university marks for the 900 who enrolled. A regression of first-year marks on GPA and test score gives R-squared 0.41, with both predictors significant. A junior analyst then adds fee-paying school to the model, finds it significant with a positive coefficient, and drafts a note concluding that fee-paying schools better prepare students. The committee notices that fee-paying school is correlated with GPA and that 300 applicants have missing first-year marks, mostly those with low entrance scores.
1. Evaluate the analyst's conclusion about fee-paying schools.
Read the worked answer
The conclusion is not supported by the analysis as run. Fee-paying school is correlated with GPA, so its coefficient in a model that also contains GPA reflects only the part of school type not explained by GPA. That is a legitimate adjusted effect, but the analyst has interpreted it as a total effect and as a causal claim, neither of which the model delivers.
Three problems compound. First, selection: attendance at a fee-paying school is not assigned, so the coefficient absorbs all differences between school types, including family background and prior opportunity, that are not in the model. Second, the correlation with GPA means the coefficient is sensitive to which other predictors are included; adding or dropping GPA could change its sign. Third, the model is fitted only to enrollers, and if enrollers differ by school type - for example if fee-paying students are more likely to enrol - the estimate is subject to selection on the outcome.
The honest statement is that, among enrollers and conditional on GPA and test score, fee-paying school is associated with a certain mark difference. Whether schools cause that difference is a separate question requiring a design that addresses selection.
2. The committee asks how to handle the 300 missing first-year marks. Advise them.
Read the worked answer
The missingness is not random: it is concentrated among low entrance scores, which are themselves predictive of first-year marks. Complete-case analysis therefore drops precisely the students most likely to do poorly, biasing the regression coefficients and inflating R-squared.
The committee should first establish why the marks are missing. If those 300 never enrolled, the correct framing is a two-stage process: model enrolment, then model marks conditional on enrolment, and be explicit that the second model applies only to enrollers. If they enrolled but their marks are unrecorded, multiple imputation is appropriate: impute marks from GPA, test score, school type, and any other available variables, generate several imputed datasets, fit the model in each, and pool the estimates with their standard errors.
Either way, report a sensitivity analysis comparing complete-case and imputed results. If the fee-paying coefficient changes substantially between the two, that instability is the headline finding and should be reported to the committee rather than resolved by choosing the more convenient result.