The Wellness Survey That Overstated Satisfaction
A regional logistics company with 1,200 employees commissions an online 'wellness and stress' survey. The survey link is sent through company email; 340 employees respond. Results show 78% report stress is 'manageable,' and the consulting firm concludes that stress is not a serious problem. Human resources, however, reports that voluntary turnover among night-shift warehouse workers-41% of the workforce-has risen 22% in the past year. Night-shift workers have limited access to company email, no company-issued phones, and only a shared break-room computer. They make up just 9% of survey respondents. The company president wants to cite the 78% figure in a press release and has asked you to review the survey before publication. The survey also offered a $5 coffee gift card to the first 100 respondents.
1. What is the most consequential sampling problem, and why does it undermine the president's proposed claim?
Read the worked answer
The core flaw is coverage bias: the sampling frame (company email) systematically excludes the night shift, which is 41% of the workforce but only 9% of respondents. Because access to the survey is correlated with shift, and shift is plausibly correlated with stress and turnover, the 78% figure describes respondents, not employees. Self-selection compounds this: those who feel least stressed may be most likely to open a wellness email, and the gift card may attract a particular subset. A large n does not fix a biased frame; 340 respondents can still be unrepresentative. The external turnover data are a warning sign that the conclusion is wrong. The president should not cite 78% as an employee-wide statistic; at most it describes the subset who could and did respond. Before any press release, the company needs a design that reaches night-shift workers on paid time.
2. Propose a revised data-collection design that would produce a defensible estimate of stress across all shifts. Explain two specific design choices and the bias each addresses.
Read the worked answer
Use stratified random sampling with shift and department as strata, then sample within each stratum so night-shift workers are represented in proportion to their share, or oversampled and weighted. This directly addresses coverage bias. Second, administer the survey during paid work time at multiple locations-break-room kiosks, paper forms in workers' languages, and a supervised 15-minute session at shift start-rather than relying on company email. This reduces nonresponse bias tied to access and literacy. Third, set a response-rate target, such as 70% overall and at least 60% per shift, track response by stratum, and send reminders. Maintain anonymity by using a drop box or independent survey administrator; night-shift workers may fear retaliation. Finally, weight results to known workforce totals by shift, department, and tenure. These choices make the estimate defensible rather than merely larger.
3. The president says, 'Even if the sample is imperfect, 340 people is a lot.' Evaluate this reasoning using the concepts of representativeness and response rate.
Read the worked answer
Sample size and representativeness are different. A random sample of 340 from 1,200 could yield a small margin of error, but only if every employee had a known, nonzero chance of selection. Here the response rate is 340/1,200 = 28%, and the frame excludes the night shift; the effective sample is a volunteer subset. The margin-of-error formula assumes probability sampling and does not repair coverage or nonresponse bias. The relevant question is not 'Is 340 a lot?' but 'Who could not be in the 340, and does that missing group differ on stress?' Since the missing group has higher turnover, the likely bias is downward. The president can say '78% of survey respondents reported manageable stress,' but not '78% of employees.' A better claim would wait for the redesigned survey.