Hybrid-Work Pilot: Confounded Productivity
Read the following summary of a pilot programme and answer the questions.
A (fictional) national tax administration piloted hybrid work in two of its six regional offices between January and June. Staff in the pilot offices worked two days a week from home; the other four offices remained fully on-site. At the end of the pilot, the pilot offices had processed 12% more cases per employee than the control offices. However, the pilot offices had also been given a simplified procedure for low-value cases in March, and 64% of their remote-day output consisted of such cases. Error rates in the pilot offices rose from 3.1% to 4.4%, while control offices stayed at 3.0%. Sickness absence fell by nine days per employee in pilot offices and by two days elsewhere. The steering group concluded that hybrid work 'does not harm productivity' and recommended extension to all offices.
1. What can properly be inferred from the 12% figure?
Read the worked answer
Only that the pilot offices out-produced the control offices - not that hybrid work caused the difference.
The text supplies a competing explanation: the pilot offices received a simplified procedure for low-value cases in March, and 64% of remote-day output was such cases. Lower-complexity cases are processed faster, so the output gap is at least partly a case-mix effect.
What is not inferable:
- that hybrid work raises productivity (confounded);
- that hybrid work lowers productivity (the figure points the other way);
- that the 12% applies to individual remote days (it is an end-of-period aggregate).
The sound inference is methodological: the comparison cannot isolate the effect of hybrid work because two variables changed at once. A candidate who picks 'hybrid work increased productivity by 12%' has accepted a causal claim the design cannot support.
2. Evaluate the steering group's conclusion that hybrid work 'does not harm productivity'.
Read the worked answer
The conclusion is not adequately supported, for three reasons.
- It defines productivity as volume only. Output per employee rose, but the error rate in pilot offices rose from 3.1% to 4.4%. Rework from errors consumes capacity; a volume measure that ignores quality can rise while effective productivity falls.
- The output gain is confounded. The simplified low-value procedure arrived in March and dominates remote-day work. Without a like-for-like case mix, the 12% cannot be assigned to hybrid work.
- The scope overreaches. Even a well-founded finding about two offices would not automatically justify extension to all six; the text gives no evidence that the other offices handle comparable caseloads.
A defensible conclusion would be narrower: 'no evidence of harm to output volume was found, but quality and case-mix effects remain unresolved.' The phrase 'does not harm' asserts an absence of harm that the data have not tested.
3. Which single piece of additional evidence would most weaken the recommendation to extend hybrid work to all offices?
Read the worked answer
A like-for-like comparison showing that, within the same case category, remote-day output per hour was no higher than on-site output - and that the error increase was concentrated in complex cases handled remotely.
Why this is the strongest choice: it attacks both pillars at once. It removes the case-mix explanation for the 12% gain (so the headline benefit disappears) and locates the quality deterioration where hybrid work would matter most. The recommendation then rests on a benefit that is not attributable and a cost that is.
Weaker alternatives:
- Staff satisfaction surveys - relevant but not decisive on productivity.
- Sickness absence data - already in the text and favourable to the pilot.
- Anecdotes from managers - not evidence of the required kind.
The test is whether the new evidence breaks the inferential link between the observed figures and the conclusion.