Attribution Dashboard That Overstates Paid Social
Your company's marketing dashboard reports spend and conversions for seven channels using a 30-day click attribution window. Last month paid social showed ROAS 6.8 while branded search showed 1.2, and the CMO is ready to move 40% of branded-search budget into paid social. The dashboard uses a dual-axis chart with revenue on the left and spend on the right, and the revenue axis starts at $200,000 rather than zero. The default date filter is 'last 7 days', but the comparison line uses the prior 30 days. Segment drill-downs include rows with as few as 12 users, shown in bright green with no confidence interval. Overall conversion rate is 2.3%, but mobile web is 1.1% and app is 4.0%, and the dashboard sums those rates without weighting.
1. Identify the specific ways this dashboard could lead to a bad budget decision, and explain the statistical or visual mechanism behind each.
Read the worked answer
The biggest risk is treating attributed ROAS as incremental ROAS. A 30-day click attribution gives paid social credit for conversions that may have happened anyway, and branded search often closes demand created by other channels. Paid social's 6.8 versus branded search's 1.2 may reflect position in the journey, not causal lift. Moving 40% of branded-search budget could reduce total revenue if branded search is defending high-intent demand.
The visual choices amplify the error. A dual-axis chart lets the designer choose scales that make revenue and spend appear correlated or divergent; the axes are not commensurable. Starting the revenue axis at $200,000 truncates the bottom of the range and exaggerates differences between channels. Comparing a 7-day default window with a 30-day prior line mixes different time periods and seasonality, so the trend is not like-for-like. Showing 12-user segments in bright green with no confidence interval invites overreaction to noise; a 12-user segment with one conversion is 8.3%, which can look extreme. Finally, summing mobile web 1.1% and app 4.0% without weighting by traffic gives an unweighted average, not the overall 2.3%; the displayed channel comparison may use inconsistent denominators.
2. Redesign the view and the metric definitions so a CMO can make a defensible budget decision. What would you show, and what would you deliberately remove?
Read the worked answer
I would separate measurement from attribution. The top of the view should state the attribution model, window, and date range once, and all channels should use the same window and the same comparison period. I would replace the dual-axis chart with small multiples or a table: spend, attributed revenue, attributed ROAS, conversion rate, and sample size per channel, each with a confidence interval. If a chart is used, both series should be on comparable scales or indexed to a common baseline, and bars should start at zero. The revenue axis should not be truncated without a clear label and a reason.
I would add an incrementality panel showing results from holdout tests or geo experiments where available, clearly labeled as causal evidence, and a 'not yet measured' label for channels without tests. Segment drill-downs should suppress or aggregate cells below a minimum sample size, for example n<50, and show intervals when n is moderate. Conversion rates should be weighted by the correct denominator; mobile web and app should be shown as separate rows with their own traffic, not summed as if they were averages.
I would deliberately remove the bright-green highlighting of tiny segments, the mismatched date comparison, and any single composite 'best channel' score. The CMO should leave with a range of plausible effects and a recommendation to test the reallocation, not a false precision number.