A Common Midterm That Predicted the Wrong Thing
Ms. Nakamura teaches CS Foundations to 28 students: nine have prior programming experience, six have IEPs with extended time and text-to-speech, and four are English learners. The department common midterm is 40 multiple-choice items written by a committee two years ago, worth 100 points, with no partial credit. The class average was 71. On the six loop-tracing items, 24 of 28 answered correctly. On the single free-response item, write a function that returns the sum of the even numbers from 1 to n, nine students produced a correct function, six of them from the prior-experience group, while nineteen wrote only a function header or left the item blank. Two items about arrays used Java syntax even though the course is taught in Python, and 21 students chose the distractor that matched Python zero-indexing. Ms. Nakamura has one department meeting to propose changes before the second common assessment is written.
1. What does the pattern of scores indicate about the assessment, and what can it not tell you?
Read the worked answer
Three findings, held with different confidence. The gap between 24 of 28 on loop tracing and nine of 28 on writing a function is the clearest signal: selected-response items reward recognizing correct code, while the free-response item requires producing it, and those are not the same construct. A test that is entirely multiple choice therefore underrepresents the standard, which is a validity problem rather than a student problem. Second, the Java-syntax array items introduced construct-irrelevant variance: students who knew Python indexing were penalized for a language they were never taught, and the distractor pattern on 21 papers confirms the items measured familiarity with Java rather than understanding arrays. Those items should be voided for everyone, not only for the student who complained. Third, the nineteen blank or header-only responses are ambiguous. That could be a production gap, or it could be time pressure, an unclear prompt, or a testing culture in which a blank costs nothing when there is no partial credit. The data cannot distinguish those explanations, and the teacher should not default to the least flattering one before checking.
2. Propose a replacement assessment design the department could adopt for the second common assessment.
Read the worked answer
Start from a blueprint rather than from item writing: list the unit objectives, then allocate items so each objective is measured at the cognitive level at which it is taught. A workable split for a 60-minute period is 20 selected-response items at 40 percent and two performance tasks at 60 percent, one scaffolded and one open. Use two-tier items, predict the output and then choose the reason, so guessing is not rewarded. Replace the single free-response prompt with a sequence: trace a given loop, complete a partial function in Parsons style, then write the function, scored with a rubric that awards separate points for loop structure, condition, boundary case, and return, so partial understanding becomes visible. Write every item in Python. Keep the accommodations the IEPs require, extended time, text-to-speech, and separate setting, because those change access rather than the construct. Have two teachers independently solve every item and flag any with more than one defensible reading before publication, and add a standing item-review step so the Java wording could not have shipped twice. A common assessment is compatible with all of this; what it is not compatible with is an unexamined item bank.
3. Given the results, what instructional moves follow, and how should the teacher raise this with the department?
Read the worked answer
In instruction: re-teach boundary conditions with a trace table and with test-first design, having students write three test cases before the function and predict each result. Use pair programming with deliberate pairing of the six successful experienced students with students who left the item blank, but have the experienced partner explain rather than type, and rotate roles. For the nineteen, diagnose before remediating: a five-minute unscaffolded prompt reveals whether the obstacle is getting started or the loop logic, and those need different supports. Build in a re-assessment on the same objectives using new items. With the department, bring data rather than complaint: present the tracing-versus-writing gap, the two Java items with distractor counts, and the blank rate, then propose the blueprint and the item-review step as shared improvements. Frame it as protecting the credibility of the common assessment, since a test that mispredicts performance will not survive parent or administrator scrutiny. Volunteer the class as the pilot site for the revised form, and offer to draft the rubric strand so the work is concrete before the next meeting.