Measurement, Attribution, and Performance Analysis
This domain tests causal reasoning: whether you can separate what the advertising caused from what would have happened anyway, and whether you know the limits of each method.
Core rules
- Build a KPI hierarchy: business outcome, marketing KPI, channel diagnostics.
- Prefer experiments: randomized controlled trials, geo holdouts, A/B tests. Pre-register the metric, estimate the sample needed, and check significance before declaring a winner.
- Attribution models - last click, first click, linear, time decay, position-based, data-driven - assign credit; they do not prove causation.
- Marketing mix modeling uses aggregate data and regression. It supports budget allocation but is weak for small channels and short-term effects, and needs calibration.
- Multi-touch attribution depends on identifiers; privacy changes and walled gardens limit coverage.
- Measure incrementality as test change minus control change, and compare it with spend.
Worked example
A geo holdout shows +8% sales in test markets and +2% in control markets. Incremental lift is 6 percentage points. If baseline revenue in the test markets was $4,000,000, incremental revenue is about $240,000. Against $300,000 of test spend, the campaign did not pay back on that measure. State the assumption that the control markets were a valid counterfactual and check seasonality before concluding.
Traps that cost marks
- Calling correlation lift.
- Ignoring seasonality or a concurrent promotion.
- Declaring a winner on a sample too small to detect the effect.
- Using last-click results to cut upper-funnel investment.
- Treating MMM output as precise rather than directional.