The 92% Satisfaction KPI
A university's annual student satisfaction survey reports 92% satisfaction with 'teaching quality', but only 340 of 3,100 students responded (11%). The institutional dashboard shows this KPI as green against an 85% target, and council members want to cite the figure in the annual report. Disaggregation shows satisfaction of 71% among international students (n=38), 68% among part-time students (n=22) and 88% among final-year students. Focus groups repeatedly mention late feedback. The quality office also holds assessment data showing a median feedback turnaround of 24 days against a policy maximum of 15 days. The survey was promoted mainly through a single email and a prize draw. The vice-chancellor asks whether the 92% can be presented as evidence of strong teaching quality. The quality office must advise on reporting and on improving the indicator.
1. Assess whether the 92% figure is fit for reporting to Council. What should the quality office advise?
Read the worked answer
The 92% is a real survey result but not a defensible estimate of overall teaching quality.
- Response bias. An 11% response rate is low, and the promotion method, a single email plus a prize draw, likely over-represents engaged or satisfied students. The 92% should be reported with n=340, response rate 11%, and an uncertainty range. The effective margin is larger than a simple random-sample calculation because of non-response bias.
- Subgroup suppression. International students at 71% (n=38) and part-time students at 68% (n=22) are small samples, so do not publish them as precise estimates; report them as signals requiring follow-up. But the gap is large enough to investigate.
- Triangulation. The 24-day median feedback turnaround against a 15-day policy directly contradicts a claim of strong teaching quality. Present both.
- Reporting language. Council can say: in a survey with an 11% response rate, 92% of respondents rated teaching quality positively; this is a positive signal but not representative of all students. Feedback timeliness is a confirmed weakness. Do not put 92% on a headline slide without the caveat.
- Decision use. Use the figure to prompt inquiry, not to close it. Ask for a follow-up stratified survey or focus groups with international and part-time students.
The quality office should advise against citing 92% as evidence of strong teaching quality and should recommend a data-quality note in the annual report.
2. Design a revised indicator set for teaching quality that would better support decision-making.
Read the worked answer
Revise the indicator set so it measures teaching quality, not only satisfaction.
Principle: use a small dashboard with perception, process and outcome indicators, each with a stated owner, frequency, disaggregation and decision rule.
Suggested set:
- Perception: overall satisfaction and a 'feedback helped me improve' item; report response rate; suppress cells with n below 30; require a minimum 25% response for institutional reporting.
- Process: percentage of assessments with feedback returned within 15 days; percentage of modules with a second marker or moderation sample; percentage of staff completing assessment calibration in the past 12 months.
- Outcome: module pass rate, grade distribution by campus and mode, resit pass rate, and equity gap between international and domestic students.
- Qualitative: termly student focus groups with a fixed sampling frame; action log with closure evidence.
- Leading indicator: assessment briefs published before term start; rubric shared with students.
Rules: no single composite score; show trend and uncertainty; disaggregate by mode, campus and student group where n is at least 30; review the indicator set annually with students. The 92% KPI can remain as one perception measure, but it should not be labelled teaching quality on its own.