Sixty-Two Percent of What?

No - it is a defensible description of 412 respondents and nothing more.

The instinct is to reach for sampling error: at n = 412, a simple random sample would give roughly plus or minus 4.8 percentage points at 95% confidence. But that arithmetic is doing work it cannot do here, for three reasons.

1. The design is not a simple random sample. Only the postcard arm was randomly selected; the online open link is self-selected and there is no way to compute a response rate for it. The two modes are pooled, so the reported figure mixes a probability sample with a convenience sample and no margin of error applies to the blend.

2. Nonresponse bias almost certainly swamps sampling error. With a 0.7% overall response, the question is not how precisely you measured respondents but how different respondents are from nonrespondents. The skew is large and on exactly the variables that predict ADU opinion: a population that is 24% over 55 is represented at 71%, and renters - the group most likely to want an ADU to rent - are represented at 9% against 38% of the population.

3. Weighting cannot rescue it. You can post-stratify to age and tenure, but a 9% renter share yields roughly 37 renters, too few to support a stable weighted estimate, and weighting corrects only for observed differences. People who care about ADUs answer ADU surveys; that attitude-based selection is invisible in the weights.

So 62% is real as a finding about engaged respondents - which is useful for understanding the arguments people will make - and unsupported as a statement about residents.

You do not have time to fix the survey, but you do have time to describe it accurately, and accurate description is the whole job here.

In the staff report:

• Report the result as a property of the sample: 'Of 412 residents who responded to an open online questionnaire, 62% supported or strongly supported ADU legalization.'
• Add a short, plain-language limitations paragraph: self-selected participation, low response relative to population, respondent profile differs substantially from ACS on age and tenure, no margin of error reported.
• Show the crosstabs you do have - support by tenure, by age, by ward - because the pattern is more informative than the topline and it demonstrates that the department knows what it is holding.
• Do not put the figure in the recommendation section. Base the recommendation on the policy analysis, code language, and the documented housing need.

To the councilmember: brief, private, non-confrontational, and specific. Explain that the number describes 412 respondents and that quoting it as citywide support will not survive the first resident who asks how many people were surveyed - and that when that happens, the substantive case for the ordinance gets damaged along with the poll. Offer an alternative sentence they can use: 'Residents who commented were supportive, and we are expanding outreach before the vote.'

Resist the temptation to say the survey 'shows' anything the design cannot support, even in a direction you favor. A weak number circulating in support of a good policy is still a liability.

Design for representativeness first and depth second, and design it so the schedule survives.

Recommended package:

• A stratified address-based random sample, not a social-media open link. Mail push-to-web with a paper option, a prepaid incentive, and two reminders. Target n of roughly 800 for about plus or minus 3.5 points at 95% confidence, and set the sample by ward so no area is invisible.
• An explicit renter oversample. Renters are 38% of the population and the group most affected; a design that yields 37 renters has not measured their view at all. Recruit through utility accounts, large apartment managers, and community organizations.
• Report the response rate and the weighting, following a standard disclosure convention, so the next survey is comparable to this one.
• Two or three focus groups plus intercept surveys at the library, transit stops, and grocery stores. This is not a substitute for the sample; it is how you learn which arguments exist and how to word the closed-ended items. Eight weeks is enough for intercepts and focus groups; it is tight but workable for one reminder cycle of a mail survey.
• Write the analysis plan before fielding. Pre-specify what subgroups you will report so the results cannot be mined for a convenient cut.

What you should decline to do: field another open online form and call it improved because the n is larger. A bigger self-selected sample is a more confident version of the same error.