Attribution dispute between paid social and email
An online homeware brand spends £60,000 a month on paid social and £8,000 on email. Last-click attribution credits social with 22% of revenue and email with 41%. The email list is built largely from social-acquired visitors who sign up for a 10% welcome discount. A holdout test pausing social for two weeks in one region showed email revenue falling 18% in that region, while total revenue fell 9%. The finance director wants to cut social by half and reinvest in email, citing the 41% figure. The brand's cookie consent rate is 58%, and 44% of purchases now happen in-app where tracking is limited.
1. Diagnose why last-click attribution is misleading here. What does the holdout test tell you?
Read the worked answer
Last-click attribution credits the final touchpoint and ignores the journey that created it. Email's 41% share is inflated because email captures people at the point of purchase - including visitors that paid social originally acquired and who signed up for the welcome discount.
Two structural problems compound this:
- Consent gap: only 58% of users consent to cookies, and 44% of purchases happen in-app where tracking is weak. So a large share of journeys is invisible, and the visible share is biased towards channels that work well in tracked browsers - typically email and direct.
- Channel dependency: email's list is built from social traffic. Cutting social starves the top of the funnel that feeds email.
The holdout test is the strongest evidence available. Pausing social in one region cut email revenue by 18% there. Email cannot cause that - social was paused, not email. This is direct evidence that email revenue depends on social. Total revenue fell 9%, which is the net effect after some demand shifted to other channels or regions.
Interpretation: social's true contribution is understated by last-click and partly hidden inside email's number. The finance director's 41%-versus-22% comparison is comparing a downstream channel with an upstream one, which is not a like-for-like decision.
Caveats I would state: one region, two weeks, and possible spillover to neighbouring regions. The test should be repeated or extended before a permanent reallocation.
2. The finance director proposes halving social spend. How would you respond, and what measurement would you put in place?
Read the worked answer
I would push back on the halving, but not defend the status quo. The proposal rests on an attribution figure the holdout test contradicts.
Argument to the finance director:
- The holdout shows social has an indirect effect on email that last-click cannot see. Halving social risks a disproportionate fall in email revenue.
- The 58% consent rate and 44% in-app share mean roughly half of journeys are unmeasured. Decisions based on the measured half will systematically favour lower-funnel channels.
- However, £60,000 a month is a large commitment with weak evidence of incremental return. That is a fair challenge, and it should be answered with testing, not assertion.
Proposed approach:
- Geo holdout tests at scale - pause or reduce social in matched regions for four to six weeks, measure total revenue, not channel revenue.
- Move to a blended measurement frame: marketing-mix modelling or incrementality testing, with last-click retained only as a diagnostic.
- Step down, don't halve: reduce social by 20% and observe total revenue and email sign-ups over six weeks. If total revenue holds, cut further; if it falls, restore.
- Track leading indicators - new email sign-ups, new-customer rate, branded search - because these show social's upstream role before revenue effects appear.
- Fix measurement gaps: server-side tracking and app attribution where possible, and consent-rate improvement.
The principle: reallocate on incremental evidence, and treat the email list as an asset that social funds, not a rival channel.