The Roster That Wasn't a Sampling Frame
A doctoral candidate is studying informal e-waste sorting in three peri-urban settlements. Site A has roughly 210 workers; the cooperative's registration list names 58 members, and the other 150-odd work informally, many of them women who sort at home. The candidate's tablet survey, run over six weeks, reached 96 workers - 61 percent of registered members but about 15 percent of the estimated total. Women are 34 percent of the roster but 12 percent of respondents. The candidate also spent four months as an unpaid record-keeper for the cooperative's buyer, a role that gave access to ledgers showing that some material is sold outside licensed channels. A funder now wants a prevalence estimate of 'average monthly income per worker' with a margin of error, and asks for it before the next grant cycle. The cooperative's chair has asked to see the raw data before it is submitted.
1. Diagnose the sampling problem and explain what the candidate's 96 interviews can and cannot support.
Read the worked answer
Three problems compound.
- Frame error: the 58-name roster is an administrative list of registered members, not a sampling frame of the workforce. The roughly 150 unregistered sorters are systematically different in income, hours, and hazard exposure, so a roster-based sample cannot represent them no matter how large it grows.
- Nonresponse and coverage: 96 interviews is 61 percent of the roster and about 15 percent of the estimated population; women at 12 percent of respondents against 34 percent of the roster signals access or refusal bias, not random loss.
- No probability mechanism: without known selection probabilities, a margin of error is not calculable. Reporting 'plus or minus 3 percent' would be fabricated precision.
Fix: build a frame by mapping workspaces and home sorting sites, then use a two-stage cluster design or time-location sampling, with respondent-driven sampling as a fallback for workers who cannot be listed. Stratify by site, sex, and registration status, report stratum-specific estimates as ranges, and label them descriptive rather than inferential.
2. How should the researcher handle the ledger information and the funder's request for precision?
Read the worked answer
The ledger is data obtained through an employment relationship, not through consented research participation, and it identifies a buyer and possibly individuals. Using it as survey data would breach informed consent - participants agreed to interviews, not to employer-record linkage - and could expose the cooperative to enforcement action.
A defensible course:
- Disclose the dual role to the funder and to participants; renegotiate consent if ledger data are to be used at all, and only in aggregate.
- Do not report off-books sales in identifiable form. Report the structure of such sales only if it can be de-identified and if retaliation risk has been assessed.
- Tell the funder plainly that the requested precision is unavailable and why, and offer an alternative: income diaries with a smaller stratified sample, or ranges from a mapped cluster sample.
Obligations to research participants outrank a contractual deadline. A funder's demand does not convert convenience data into a prevalence estimate, and saying so is part of the researcher's job.
3. What does the record-keeper role imply for reflexivity and for the validity of the findings?
Read the worked answer
Four months as an unpaid record-keeper moved the researcher from observer to participant with a stake in the cooperative's reputation.
Benefits: trust, access, and tacit knowledge of how income is pooled - genuinely useful for interpreting the survey.
Costs:
- Role conflict: workers may have felt obliged to answer, compromising voluntariness.
- Selection: the buyer's associates are over-represented in the network the researcher can reach.
- Interpretive capture: the ledger view anchors income estimates toward recorded transactions and misses informal transfers.
Mitigations: keep a reflexive journal separating observations made in each role; have a colleague interview workers the researcher supervised; record consent separately for each mode of participation; describe the role in the methods section; and test interpretations against data gathered by someone without the record-keeper relationship. Reflexivity here is not confession - it is a validity procedure that tells readers how the sample and the numbers were shaped.