The Report That Answered the Wrong Question
A retail chain's executive team asks for a report on "why store sales dropped 8% last quarter." The analyst delivers a 30-page report showing that footfall fell 6%, conversion rose slightly, and average basket size was flat, with a recommendation to "improve marketing." The COO says the report is "interesting but doesn't tell me what to do." Two regions account for most of the decline: one has a new competitor opened 400 meters from three stores, the other had a distribution problem that left shelves empty for 11 days. The analyst had this data but did not segment by region. The next executive meeting is in five days.
1. Explain what went wrong in the analysis and how to restructure it for the next meeting.
Read the worked answer
The analysis answered the chain-level question and missed the decision-level question. Aggregation hid the two real causes.
What went wrong:
- The request was accepted as stated ("why did sales drop") without clarifying what decision the executive team needed to make. A cause-level question needs a segmentation plan.
- Region-level data existed but was not used. Averaging a competitor shock and an 11-day stockout produces a bland national story.
- The recommendation ("improve marketing") is not actionable because it is not tied to a cause.
Restructure: Lead with a one-page answer: the 8% decline is concentrated in two regions with distinct causes; the rest of the chain is roughly flat. Show a contribution waterfall (competitor region, stockout region, remainder). For each cause, state the evidence, the estimated sales impact, and a decision option with owner and timing. Put method and caveats in an appendix. Five days is enough for this if the data already exists, which it does.
2. Write the clarifying questions you would have asked at the start, and explain how each would have changed the work.
Read the worked answer
- What decision will this report support? If the answer is "where to intervene," the analysis must be segmented by region and cause, not national averages. This single question changes the structure.
- What is the unit of analysis and the comparison baseline? Same quarter last year, prior quarter, or budget? Each yields a different 8%. Clarifying prevents a number dispute at the meeting.
- Which segments matter? Region, store format, product category, channel. Asking this would have surfaced the two-region concentration early.
- What data do we have on external and operational factors? Competitor openings, stockouts, promotions, weather. This would have prompted the distribution and competitor data to be pulled into the first pass.
- What would make you change your current plan? This tests whether the report can actually influence a decision or is being commissioned for reassurance.
- Who will challenge this, and on what grounds? Anticipates the COO's "so what" reaction and builds the answer into the first page.
Ask these before analysis, not after. A 15-minute scoping conversation would have saved most of the 30 pages.