When the Test Doesn't Measure the Standard
At a comprehensive high school, the biology department requires a common unit assessment for all 31 students in each section. The ecosystems test has 18 multiple-choice items, four multi-step word problems with long reading stems, and two constructed-response items. The four students with reading-related IEPs and five English language learners averaged 54% on the test, while the class average was 71%. Item analysis shows that most of these students answered the factual multiple-choice items correctly but missed the word problems that required identifying relevant data from a paragraph. Several wrote scientifically accurate constructed responses with spelling and grammar errors. The department allows one retake within two weeks and requires that the retake be equivalent in format. Mr. Patel, the teacher, believes the test is measuring reading endurance more than ecosystem reasoning, but he does not want to lower expectations or violate the common-assessment policy. He has 31 students and one 50-minute period for the retake.
1. Evaluate the validity and fairness of this assessment. Explain what the subgroup score pattern does and does not tell Mr. Patel about student learning.
Read the worked answer
The pattern suggests a construct-irrelevant variance problem: the test may be measuring reading endurance as much as ecosystem reasoning.
- Validity: The four word problems have long reading stems. Students who know the science but decode slowly or lack academic English are penalized on items intended to assess data interpretation. If the standard is ecosystem reasoning, the reading load is not the target.
- Fairness: The same pattern across IEP and ELL groups is an opportunity gap, not necessarily a knowledge gap. Averaging 54% while getting factual items correct is consistent with language and working-memory demands.
- Reliability of inference: Constructed responses with accurate science but spelling and grammar errors should be scored with a science rubric that separates ideas from conventions. If conventions are double-counted, scores understate learning.
- Policy tension: The common assessment and retake policy are legitimate for comparability, but equivalent in format does not mean identical reading load. Equivalent rigor can be maintained with different access features.
- What the data do not show: The scores alone do not prove students lack ecosystem knowledge, nor do they prove the test is invalid for every student. They justify an item-level review.
2. Propose a revision to the assessment and a plan for the retake that maintains the department's standards and comparability while improving access.
Read the worked answer
Revise the assessment and the retake so both measure the same standards.
- Audit items against standards: For each word problem, identify the science construct and the reading demand. Replace unnecessarily complex stems with concise language while keeping the data interpretation.
- Add access features to the common test: Provide read-aloud for students with reading IEPs when decoding is not the target, chunked text, a glossary of non-content terms, and a graphic organizer for constructed responses. These do not change the science standard.
- Use a two-part retake: Part A contains core multiple-choice and short data-interpretation items. Part B is a performance task where students analyze a graph or data table and write a claim-evidence-reasoning paragraph. Score with a science rubric that separates scientific accuracy from conventions.
- Keep comparability: Use the same standards, scoring rubric, and blueprint for all sections. Document the accommodations so the department can review equivalence.
- Do not simply make the retake easier: If the retake only lowers reading load, it is appropriate; if it removes data interpretation, it is not equivalent.
3. Describe feedback and grading practices that would keep the assessment focused on science learning rather than language conventions.
Read the worked answer
Feedback and grading should communicate science learning, not punish language.
- Rubric: Use a four-point science rubric covering claim, evidence, reasoning, and use of data. Report conventions separately rather than folding them into the science score.
- Feedback: On constructed responses, write one science-focused comment and one language-focused comment. For ELLs, focus on the target academic language, such as therefore or the data show, rather than every grammar error.
- Retake policy: Require students to complete a brief error analysis before the retake. This makes the retake a learning opportunity, not a reset. Apply the same score-replacement policy consistently.
- Gradebook: Separate product, process, and progress where possible. If the common assessment remains one grade, weight it appropriately and use other evidence, such as lab notebooks and oral explanations, so a single reading-heavy test does not control the grade.
- Transparency: Show students the rubric and annotated exemplars. Explain that the retake assesses the same standards with accessible language.