Attribution Fight: Paid Social vs. Brand Search
An online retailer spends $500,000/month across paid social and paid search. Last-click attribution credits paid search with 62% of conversions and paid social with 18%, so the team proposes cutting social by half and moving the budget to search. However, a geo holdout test run last quarter showed that regions where paid social was paused saw total conversions drop 14% versus control regions, while search-paused regions saw only a 3% drop. Brand search spend is substantial, and many brand-search clicks come from users who already know the brand. The CFO wants a defensible reallocation and is wary of both last-click and vendor-supplied lift studies.
1. Explain why last-click attribution and the geo test tell different stories, and which should carry more weight for a reallocation decision.
Read the worked answer
Last-click credits the final touchpoint, so brand search - which captures users already persuaded and often navigates directly - absorbs credit for demand created upstream by social. Paid social typically works earlier in the funnel, so it looks weak under last-click even when it drives incremental demand. The geo holdout is an incrementality measure: it compares total outcomes with and without the channel, holding other factors roughly constant, so it captures the causal contribution rather than the credited share. For a reallocation decision, the geo test should carry more weight because it measures incrementality, while last-click measures credit assignment. The 14% total-conversion drop when social was paused versus 3% for search suggests social is more incremental than its 18% credit implies, and search is less incremental than its 62% credit implies.
2. The CFO distrusts vendor lift studies. How would you design a reallocation test that produces credible evidence?
Read the worked answer
Run a randomized geo experiment (or conversion-lift holdout) designed in-house or with an independent measurement partner: randomly assign matched geos to treatment (reduced social, increased search) and control (status quo), pre-register the metric (total conversions/revenue, not channel-attributed), run long enough to cover purchase cycles, and check pre-period comparability. Include a brand-search-specific holdout to isolate its true incrementality. Control for seasonality and external shocks via the control group. Analyze intent-to-treat and report confidence intervals. Use the result to set budget by incremental return per dollar, not by credited conversions. Avoid relying on vendor-supplied lift without access to methodology and raw data.
3. Suppose the test confirms social is highly incremental but search is mostly capturing existing demand. What is the defensible reallocation, and what caveat should accompany it?
Read the worked answer
Shift budget toward the channel with higher incremental return per dollar - here, protect or increase paid social and trim the least incremental search, especially brand search that mostly harvests existing intent. Caveat: search may still serve a defensive role (competitor conquesting, protecting against rivals bidding on your brand) and some brand-search spend may be non-negotiable for compliance or presence; cutting it entirely can cede ground. Also, incrementality can vary by segment and season, and social's incrementality may diminish at higher spend (diminishing returns). Recommend phased reallocation with ongoing holdout measurement rather than a one-time swing, and re-test after major changes.