Performance Audit When the Program Measures Outputs Only
The Supreme Audit Institution is conducting a performance audit of the ‘Digital Rural Schools’ program, which spent 12.4 billion tenge in 2023–2025 to equip 1,200 rural schools with computer classrooms and internet access. The ministry’s annual report states that 98% of schools received equipment and calls the program ‘highly effective.’ Audit fieldwork finds that 37 schools received computers but have no working internet connection because local telecom infrastructure was not ready; the program’s approved indicators measure only ‘number of computers delivered’ and ‘share of schools with equipment installed.’ There is no baseline for student digital literacy, and the ministry says that without an outcome indicator the auditors have no valid criteria. The audit team also finds that 14 schools received equipment before their construction was completed. The audit director must decide how to frame criteria and whether to report an effectiveness finding.
1. Formulate the audit criteria and audit questions you would use, and explain why the ministry’s ‘no outcome indicator’ objection does not by itself prevent a finding.
Read the worked answer
Criteria should come from the program’s approved budget passport, state audit and budget legislation, and the ministry’s own service standards. Useful questions: Did the program define outputs and outcomes that match its objective? Did schools receive both equipment and connectivity before the program reported coverage? Were payments made only for functionally complete classrooms? Are indicators measurable, baseline-based, and verifiable?
The absence of an outcome indicator is itself a finding about program design and performance measurement. It is not a reason to avoid an effectiveness conclusion. The auditor can use the program’s stated objective (digital access) and its approved coverage definition. If the ministry reported 98% coverage using ‘equipment delivered,’ the auditor can test whether that indicator is a valid measure of the objective. The 37 schools show an output delivered without the service; the 14 schools show delivery before readiness. Both are testable against criteria of completeness, economy, and program design.
2. Classify the 37-school and 14-school situations as findings or observations, and state what additional evidence you need before concluding on effectiveness.
Read the worked answer
The 37 schools should be treated as a performance finding on effectiveness and indicator validity if evidence confirms no connectivity and no alternative access. The 14 schools should be a finding on compliance and internal control if equipment was delivered before construction completion and accepted without readiness.
Before concluding, obtain site verification, telecom activation records, acceptance certificates, payment records, and the program’s definition of ‘school equipped.’ Check whether the 37 schools were counted in the 98% and whether management knew about the gap. Do not conclude that the whole program was ineffective without a representative sample; report the exceptions, their cause, and the measurement weakness. A finding should state condition, criteria, cause, effect, and recommendation.
3. Draft the recommendation logic for correcting the indicator system and following up on the exceptions.
Read the worked answer
Recommend that the ministry revise indicators to include connectivity, classroom readiness, and a baseline and outcome measure; require joint acceptance by education and telecom authorities; and reconcile reported coverage to activation records. The recommendation should be addressed to the ministry with a deadline and a responsible official.
Follow-up should test whether the revised indicator system was approved and whether the 37 schools were connected. The auditor should also verify whether payments for the 14 incomplete schools were adjusted. The recommendation logic should link the design weakness to the reported overstatement of performance, so that management cannot treat the issue as a mere reporting error.