Night-Shift Operators and a Survey That Says Everything Is Fine
The warehouse order-entry system is being replaced. It is used by 180 operators across three shifts, including a night shift running 22:00–06:00 that cannot attend daytime workshops. A survey sent to 60 operators returned 22 responses, mostly from day shift, and concluded that the current system is fine. That contradicts more than 40 incident tickets a month about barcode scanning failures and duplicate orders. Two shift supervisors give conflicting accounts of the workaround: one says operators key SKUs manually when scanning fails, the other says they write orders on a clipboard and re-enter them later. The union agreement requires consultation on any change to operator workflow. Meanwhile, managers attended a vendor demo last month and now speak about the replacement in terms of that demo's screens and features, although no agreed requirements exist. Your elicitation plan is due to the program board in two weeks.
1. Critique the survey as an elicitation technique in this situation and set out a revised plan for reaching night-shift operators and supervisors.
Read the worked answer
Why the survey is weak here:
- Sampling bias. 22 of 60 responses, mostly day shift, cannot represent 180 operators across three shifts. Night shift is the group most likely to hit scanning problems and is effectively excluded.
- Wrong instrument for the question. A survey measures stated opinion; you need observed behaviour and exception handling. "The system is fine" is a plausible answer to a badly framed question and is contradicted by operational evidence.
- No reconciliation with data. Forty-plus tickets a month is roughly 480 a year against 180 operators - that is not background noise and should have been tested in the survey design.
Revised plan:
- Observation on the floor across at least two night shifts and two day shifts, using contextual inquiry: watch a scan fail and record what happens next.
- Incident and log analysis: scan failure logs, manual-entry audit trails, duplicate order reports by shift and site, and correction timestamps.
- Short intercept interviews at shift handover, with time-in-lieu or paid time, rather than pulling operators off the line.
- Separate interviews with each supervisor, then a joint session with the evidence on the table.
- Union representative involvement from design stage, so consultation is real and documented.
Present the plan with an explicit sampling rationale: which shifts, sites and operator profiles, and how many of each.
2. The two supervisors contradict each other about the workaround. How would you resolve the contradiction, and what evidence would make you confident the process is genuinely understood?
Read the worked answer
Treat the contradiction as data, not noise. Supervisors often describe the official or preferred workaround rather than every variant. Both accounts may be true at different sites, shifts or failure types.
Resolution approach:
- Interview each supervisor separately and ask them to walk through a specific recent failure with timestamps and order numbers.
- Pull system evidence for those orders: scan-failure logs, manual keying timestamps, duplicate order flags, correction history.
- Observe several nights and at least two sites, because a single site may mask a local variant.
- Ask operators, not only supervisors, and ask them to demonstrate rather than describe.
- Hold a joint session where each account is checked against the artefacts, so the conversation moves from opinion to evidence.
What would give confidence:
- An as-is process model with named variants, the conditions that trigger each, and estimated frequency per shift.
- Reconciliation between the modelled exception volume and the incident ticket rate.
- The workaround reproduced during observation, not just reported.
- Operators confirming the model in their own words, and supervisors accepting it.
If the variants differ by site, treat that as a design input - one workaround may need to be supported while the other is retired deliberately.
3. Managers are anchored on the vendor demo and the union requires consultation. How do you manage both without stalling the schedule?
Read the worked answer
Anchoring: the demo has become the reference point before the problem is defined. Counter it by shifting the conversation from screens to outcomes.
- Run a short session where managers state what must improve - duplicate orders, scan reliability, time per order - and record these as measurable objectives.
- Capture anything derived from the demo explicitly as a requirement with source "vendor demo", then validate it with operators. This respects the managers' input without letting it bypass elicitation.
- Present the demo as one candidate design among others, and show where operators' observed needs are not addressed by it.
- Avoid arguing that the demo is wrong; ask which operator problems each feature solves.
Union consultation:
- Engage the representative at plan stage, share the observation and interview approach, and agree how findings are attributed.
- Clarify early what counts as a change to workflow, so consultation is scoped rather than open-ended.
- Build consultation milestones into the plan, in parallel with elicitation, not after design.
- Keep individual operator input confidential in reporting; report themes and volumes.
Schedule protection: set decision gates - objectives agreed, as-is confirmed, requirements baselined - and state what happens if the union consultation milestone slips, so the risk is visible to the board rather than absorbed quietly by the team.