The Missing 38 Percent: Building a Priority Framework on Uneven Data

The core problem is coverage bias, not sample size. The 38% who did not respond are not a random slice: alternative-school students and chronically absent students have substantially higher substance use rates than the general student body. That means 29% should be read as a floor, not a point estimate, and the true figure could be several points higher.

Equally important, the community forum is self-selected and demographically skewed (median age 61, two of five townships). It is qualitative data about what engaged adults are worried about, not prevalence data about youth. Citing it alongside the survey as if the two were comparable is a category error.

Four weeks of concrete steps:

1. Administer the survey (or an abbreviated version) at both alternative schools.
2. Add a brief absentee follow-up, by phone or on return, for students with 10+ absences.
3. If enrollment and attendance data allow, post-stratify or weight by grade and school type.
4. Code the forum notes thematically and present them as themes, never as percentages.
5. Add archival indicators with age breakouts: juvenile justice referrals, emergency department mentions, treatment admissions by age band.
6. Write the limitations into the plan itself, so the funder sees the reasoning rather than a falsely confident number.

A single indicator should not drive a three-year plan. Use a documented criteria matrix scoring each substance on: magnitude (prevalence), severity (consequences, including acute harms), trend, community readiness, existing prevention infrastructure, strength of available evidence-based strategies, and equity impact.

Applying it here: alcohol likely scores higher on magnitude and on the strength of environmental strategies (availability, pricing, social host, enforcement). Methamphetamine scores higher on trend and on infrastructure gaps. Note that treatment admissions measure treatment-seeking and system capacity as much as incidence; a rising admission count does not prove rising use, but a rising trend with no prevention infrastructure is a legitimate reason to act.

A defensible outcome is a plan with two priorities and two different intervention logics: universal environmental strategies for alcohol (retailer compliance, social access, parent norms) and selected/indicated strategies plus workforce and treatment linkage for methamphetamine.

What I would say to her: alcohol is very likely our largest problem by volume, and the survey number is probably an underestimate; but naming only alcohol would leave a fast-moving indicator with no infrastructure unaddressed, and our funder will ask why. Bring the matrix to the full coalition, record the vote and any dissent, and document the rationale in an appendix.

Those are process and output measures, and they cannot distinguish a coalition that changed youth behavior from one that kept 400 adults in chairs. Worse, they are easy to hit and will drive staff behavior toward filling rooms rather than toward the outcomes in the plan.

Replace them with a three-tier measurement structure:

• Outcome measures tied to objectives, e.g. reduce past-30-day alcohol use among 10th graders from 29% to 24% by Year 3; reduce retailer non-compliance from X% to under 10%.
• Intermediate measures on the causal pathway: perceived risk of harm, parental disapproval of underage drinking, perceived availability, social access sources.
• Process measures that matter: reach into the priority population (not raw attendance), and fidelity/adherence for any curriculum delivered.

Two planning constraints to design around: the school survey runs on a biennial cycle, so you get at most two measurement points in three years (build in a lightweight interim measure); and retailer compliance data depend on the enforcement agency's schedule, so negotiate data-sharing now, not in Year 3. Every measure needs a named source, frequency, and person responsible.