A question that cannot distinguish two opinions
A research team drafts an item asking whether the government should reduce local taxes and expand a community health service. Respondents must select one agreement score, although several pilot participants explain that they support one policy and oppose the other. The project manager initially treats these comments as a nuisance and proposes keeping the question because it is short. The survey’s stated purpose is to measure preferences for each policy separately, not a combined package. There is room in the questionnaire for additional items, and the data have not yet been collected. The analyst must decide how to revise the measure, test the revision and describe what an eventual response score would mean.
1. Identify the measurement problem.
Read the worked answer
The item is double-barreled: one score combines two potentially different preferences. It cannot distinguish the attitudes required by the research objective.
2. How should the revision be checked?
Read the worked answer
Ask separate clearly worded questions and pilot them with relevant participants. Check interpretation, response choices and flow, rather than assuming that a shorter item is necessarily a better measure.