When the Survey Speaks for the Wrong County
A county health department (210,000 residents) is finalizing its community health needs assessment with six weeks left and no remaining budget. Three data sources disagree. A mail/web survey drew 1,240 responses (14% response rate); respondents were 62% age 55+, 78% homeowners, and 81% White, while the county is 48% White with a median age of 37. Emergency department data show asthma visits in two ZIP codes running 2.4 times the county rate, and those ZIP codes are majority renter-occupied. Three focus groups with 22 Spanish-speaking residents named mold, pests, and lack of transportation as their top concerns. The board chair has asked staff to publish the survey's ranked “top 10 needs” as the official findings and move directly to planning.
1. Should the survey rankings become the official priority list? Explain how you would combine the three sources and what you would say to the board chair.
Read the worked answer
Publish the survey rankings as one input, not as the priority list.
Reasoning:
- The 14% response rate is not neutral missingness. Older homeowners are heavily overrepresented (81% White vs. 48% county; 78% homeowners). The people most likely to report mold, pests, and transportation barriers - renters and younger, non-White residents - are the people the survey is structurally least able to hear.
- The qualitative and quantitative sources converge: the two high-asthma ZIP codes are majority renter-occupied, which is exactly the housing profile the focus groups described. Convergence across independent sources is what raises confidence, not sample size alone.
- A survey ranks salience to respondents, not burden in the population.
What to do: score each candidate need against explicit criteria - magnitude, severity, trend, equity impact, and whether an effective intervention exists - using the best available data. Asthma/housing and transportation will score high on severity and equity; the survey's top items may score high on magnitude only. Present both rankings side by side rather than substituting one for the other.
To the board chair, avoid arguing that the survey is bad. Frame the consequence: if the plan's priorities mirror a sample that misses the highest-burden ZIP codes, the plan will neither move the county's worst outcome nor survive scrutiny from funders and community partners. Offer to keep the survey findings in the report as a fully documented source.
2. What limitation and data-quality issues must be documented, and what low-cost steps can you still complete in six weeks?
Read the worked answer
Document (in the report, in plain language):
- Response rate, demographic comparison to ACS estimates, and the direction of likely bias (underrepresentation of renters, younger adults, and Latino residents).
- Focus groups: n=22, purposive, not generalizable - signal, not estimate.
- ED data measure utilization, which is confounded by access and insurance; high ED rates can reflect poor primary care access as much as high prevalence.
- No pediatric or employer-based data in the current set.
Feasible in six weeks at no cost:
- Post-stratification weighting of the survey to ACS age/race/tenure distributions and re-run the rankings - if the weighted and unweighted top items diverge, that divergence is itself a finding.
- Eight to ten key-informant interviews by phone (school nurse, EMS director, two property managers, a pediatrician, a food pantry director) to test the housing hypothesis.
- Request existing secondary data partners already hold: school district absenteeism and asthma-action-plan counts, EMS call data by ZIP, Medicaid claims summaries.
- Ask partner agencies to push the survey link specifically to renters and Spanish speakers; label any late responses as a supplemental non-probability sample.
Be honest that representativeness cannot be repaired in six weeks. The report should state that the survey estimates are indicative and that findings are triangulated, and should recommend a redesigned sampling approach for the next cycle.