Item Analysis and a Draw-and-Tell Prompt in Pre-K
Ms. Whitfield's pre-K team wrote a 12-item multiple-choice end-of-unit check on life cycles (butterfly, frog, plant) for 22 four-year-olds, plus one constructed-response prompt: 'Draw and tell one thing that happens first when a seed grows.' After administration, the team reviews item data: Item 3 (a picture of a caterpillar and a butterfly, 'Which comes first?') was answered correctly by 20 of 22 children. Item 7 ('What does a seed need to grow?') had a distractor 'sunlight' chosen by 14 children; the keyed answer was 'soil.' Item 9 used a photo of a frog's eggs with the word 'spawn'-most children chose 'baby fish.' The constructed-response prompt produced 12 drawings of a tree, 5 scribbles, and 5 'I don't know.' One child with an IEP for language impairment did not respond.
1. Analyze the three flagged items. What is wrong with each, and how would you revise them?
Read the worked answer
Item 3 has a p-value of .91 (20/22), so it is too easy and provides little information; keep it only as a warm-up or replace it with a similar item that asks children to order two stages. Item 7 is a flawed key: sunlight is a legitimate need for seed growth, so the 14 children who chose it may be scientifically correct, not confused. The item penalizes accurate knowledge and measures test-taking rather than life-cycle understanding. Revise to a single clear construct, e.g., 'Which of these does a seed need to sprout?' with one correct answer and plausible foils that are not also correct, or use a picture sort. Item 9 uses 'spawn,' a vocabulary word that many 4-year-olds do not know, and the photo may be ambiguous; it measures vocabulary and prior experience, not life-cycle knowledge. Replace 'spawn' with 'eggs' and use clear, realistic pictures. Across the test, read items aloud, use one idea per item, and avoid 12 items in one sitting for 4-year-olds.
2. Interpret the constructed-response results. What does the pattern suggest, and how should the team design the prompt and scoring?
Read the worked answer
The 12 tree drawings suggest the prompt was ambiguous: 'when a seed grows' can be heard as 'what a seed grows into,' so children answered a different question. The 5 scribbles and 5 'I don't know' are not evidence of no knowledge; they may reflect unfamiliar test format, fine-motor demands, or lack of scaffolding. For 4-year-olds, drawing plus oral explanation is appropriate, with an adult scribing the child's words. Revise the prompt to a concrete sequence: 'Draw what happens first, next, and last when we plant a seed,' and provide a three-box frame, picture choices, and a modeled example with a different plant. Score developmental levels, not correctness: no response, isolated marks, recognizable parts, sequence with support, sequence independently with oral explanation. Accept home-language responses and scribe them. Keep a portfolio sample and an observation note. The team should reteach with real planting and sequencing cards before reassessing.
3. One child with an IEP for language impairment did not respond. What should the team do, and what cautions apply to the overall data?
Read the worked answer
First, treat the nonresponse as assessment information. The child may need accommodations: adult scribing, picture selection instead of drawing, extra time, a familiar setting, home-language support, or a break. The IEP team should review the accommodation and ask the special educator or SLP to observe. Do not score the blank as 'no knowledge.' Offer an alternative mode-pointing to sequence cards, arranging felt pieces, or explaining orally-and document what was provided. More broadly, the team should not overhaul the life-cycle unit or retain children based on one flawed 12-item check. Item 7 had a defective key, item 9 measured vocabulary, and the constructed response was ambiguous, so the data are not valid for high-stakes decisions. Use the results to revise items, reteach with concrete experiences, and collect multiple sources: observations, work samples, and family input. The purpose of assessment in pre-K is to inform instruction and identify supports, not to rank 4-year-olds.