Outsourced assessment and freelance trainers at an IT academy
Meridian IT Academy delivers vendor certification courses to 1,200 learners a year, mostly funded by employers. It employs six full-time trainers and twelve freelance trainers. The academy recently outsourced its online assessment platform to a vendor that hosts question banks, schedules exams and calculates scores. Trainers mark practical assignments; there is no moderation or second marking. In the past quarter, 47 results had to be recalculated after the vendor silently changed a scoring algorithm. Three employer clients complained about inconsistent marking between trainers. The academy has no record of competence for six of the freelance trainers, two of whom delivered 14 courses in the last year. Course materials are stored on trainers' personal drives, and one trainer shares learner assessment records through a personal messaging account. The academy wants to include assessment and certification in its ISO 21001 scope. The operations manager says the vendor is responsible for the platform and freelance trainers are responsible for their own competence.
1. Identify the main implementation gaps relative to operational control, competence and externally provided processes, and propose prioritized controls.
Read the worked answer
Main gaps are competence under 7.2 for freelance trainers, operational control of assessment under 8.5, control of externally provided processes under 8.4, documented information under 7.5, and release of results under 8.6. Prioritized controls: define competence requirements for trainers and assessors and verify and record evidence for all 18 trainers before further deployment; introduce assessment rubrics and moderation sampling, for example second-mark a sample of practical assignments and review discrepancies; require the vendor by contract to notify changes and validate algorithm changes before release; verify vendor results against a sample; move learner records to approved systems; and establish an appeals and correction process. The 47 recalculated results show the risk of unverified external change. Controls should be proportionate: not all assignments need double marking, but a risk-based sample and clear criteria do.
2. Assess the operations manager's statement about responsibility. What must the academy do about the vendor change and the freelance trainers?
Read the worked answer
The statement is wrong in both parts. Outsourcing a process does not outsource responsibility for conformity; 8.4 requires control of externally provided processes, products and services, including criteria for selection, monitoring and re-evaluation. The academy should have a contract clause requiring notification of scoring changes, a change-control process, acceptance testing and a fallback. For freelance trainers, the academy remains responsible for ensuring the competence of persons doing work under its control; it must determine requirements, verify competence, and retain records. The two trainers who delivered 14 courses without evidence should be assessed before further deployment, and the courses already delivered should be risk-assessed for impact on results. This is not a blame exercise; it is a control gap.
3. Specify the documented information the academy should create or retain for assessment and certification, and justify each item.
Read the worked answer
Documented information should include the trainer and assessor competence matrix with qualifications, experience and observed assessments; assessment criteria and rubrics; moderation and second-marking records with discrepancy logs; the vendor contract, service levels and change notifications; validation and acceptance test records for platform changes; learner results, appeals and correction records; records of learner identity and consent; data protection and retention rules; and nonconformity and corrective action records. Each is needed to provide evidence of conformity, protect learners, and support traceability. Clause 7.5 requires documented information required by the standard and by the organization; the test is whether the information is necessary for the effectiveness of the EOMS, not whether it is convenient.