Self-Evaluation That Finds What the Dean Wants
A faculty of education is preparing its six-yearly programme self-evaluation. The self-evaluation team is chaired by the dean and includes three programme leaders, one student representative and no external member. The evidence pack reports 88% student satisfaction from a survey with a 41% response rate and a 91% overall pass rate. Progression data, however, show that 24% of first-year students failed one core module and the resit pass rate is 58%. Several staff mention that the satisfaction survey was promoted during a compulsory lecture. The dean wants to submit a 'mature' rating with no major weaknesses and has asked the team to 'keep the narrative positive'. A junior quality officer notices that the draft report omits the progression data and quotes only the satisfaction figure. The report is due in 10 days; an external review panel visits in eight weeks. The officer must decide what to raise and how.
1. Identify the main threats to the credibility of this self-evaluation and recommend concrete changes before submission.
Read the worked answer
Credibility is threatened by four things: independence, sampling, selective evidence and missing triangulation.
- Independence. A dean chairing the team that judges the dean's faculty is a conflict of interest. Either appoint an independent chair, such as a senior academic from another faculty, or have the dean act as a witness rather than chair. Add an external critical friend to read the draft.
- Sampling bias. A 41% response rate is not fatal, but promotion during a compulsory lecture and a prize draw make the 88% a weak estimate. Report the response rate, the promotion method and an uncertainty range; do not present 88% as the view of all students.
- Selective evidence. Omitting the 24% first-year failure rate and 58% resit pass rate is not a positive narrative; it is an evidence gap. Include them, disaggregate by module and mode of study, and explain what the faculty has already tried.
- Triangulation. Add focus groups with students who failed, module leader interviews and a review of resit support. Compare survey, progression and assessment data.
- Judgement discipline. Use a rubric that separates 'meets' from 'mature' and require evidence for each claim. A 'mature' rating with an unexplained 24% failure rate will not survive an external panel.
Concrete pre-submission changes: rewrite the evidence table to include all indicators, add a limitations section, hold one validation meeting with an independent chair, and attach an action plan for the resit issue.
2. How should the quality officer respond to the dean's instruction to keep the narrative positive and omit the progression data?
Read the worked answer
The quality officer should not comply silently, but should avoid turning it into a personal confrontation.
- Reframe the request. Say the aim is a credible submission, not a negative one. Omitting known data creates a foreseeable risk: the external panel will see progression data in the institutional dataset, and a later discovery would damage the faculty more than an honest weakness.
- Put it in writing. Send a short note to the dean and team: the draft currently omits the first-year failure and resit data; including them with an improvement action is recommended because the panel has access to the same data and may test the claim of maturity. Keep it factual.
- Offer a solution. Draft the paragraph and the action plan so the dean is not being asked to accept a bare criticism. Show that reporting a weakness with evidence can support a 'mature' judgement if the response is credible.
- Escalate if instructed to falsify. If the dean insists on deleting accurate data, the officer should refer to the institution's code of conduct and quality policy, and escalate to the registrar or audit committee. Do not alter the underlying data.
- Document. Keep the draft versions and the advice given. Transparency protects the officer and the institution.