Assessment and Professionalism
Assessment and Professionalism tests how to measure language growth defensibly and how to act as an ethical member of the profession. Expect questions on formative and summative assessment, validity and reliability, rubrics, item analysis, proficiency-oriented assessment, feedback, confidentiality, copyright, and professional organizations and development.
Core concepts
- Validity: the assessment measures the intended construct. A grammar quiz cannot certify interpersonal speaking.
- Reliability: consistency across raters, tasks, and occasions. Rubrics with descriptors and anchor samples raise inter-rater reliability.
- Formative assessment informs teaching; summative assessment reports achievement. A single end-of-unit test is summative and should not be the only evidence.
- Alignment: objectives, activities, and assessments must target the same mode and level.
- Item analysis: difficulty (p-value) and discrimination. A distractor chosen more often by high scorers signals ambiguity, not student weakness.
- Ethics: confidentiality of records, copyright and fair use for authentic materials, appropriate use of student work, and mandated reporting.
Worked example
On a listening item, 70% of the top third and 30% of the bottom third chose option B, and the item's discrimination index is negative. What should you do? Reasoning: high achievers outperforming low achievers on a distractor means the item rewards a misconception or is ambiguous. Inspect the audio and the wording, revise or discard the item, and do not use it for grading. The problem is item quality, not student ability.
Traps that cost marks
- Treating reliability and validity as synonyms.
- Averaging participation or behavior into an academic language grade without criteria.
- Using one modality to assess a standard written for interpersonal communication.
- Sharing student scores or accommodation information informally.
- Calling a test proficiency-based when it samples memorized vocabulary.