The Dashboard Dilemma: When Metrics Don't Match
A financial services company has a dashboard that shows 'Total Revenue' from their Adobe Analytics data. The dashboard uses a calculated metric that sums 'Revenue' (the standard e-commerce metric) and 'Event 10' (a custom event they use for application submissions, which they also count as revenue). The marketing team reports that the dashboard shows $1.2M for the last month, but the finance team's records show $1.5M. The analytics team investigates and finds that the 'Revenue' metric includes tax and shipping, while finance reports net revenue. Also, Event 10 is not always set with a value; sometimes it's just a counter. The dashboard is used by executives to make budget decisions.
1. What are the likely causes of the discrepancy, and how would you resolve it?
Read the worked answer
The discrepancy has multiple causes: 1) 'Revenue' in Adobe Analytics includes tax and shipping (if passed in the products variable or set in the revenue field), while finance reports net revenue. 2) Event 10 is sometimes set without a value, so summing it adds zero for those instances, but if it's a counter, it might be counted as 1? Actually, if event10 is a numeric event, it only adds value when set. If it's a counter, it increments by 1 but may not have a revenue value. The calculated metric may be incorrectly summing event10 as if it were revenue. 3) The calculated metric might be adding event10 instances (count) to revenue (currency), which is not meaningful. 4) Data latency or missing data. To resolve: 1) Clarify definitions with finance: decide on a standard revenue definition (gross vs net). 2) Adjust the implementation to pass net revenue separately (e.g., use a custom event for net revenue) or use a classification. 3) Fix event10: ensure it always has a value if it's meant to represent revenue, or use a separate event for application submissions and don't mix with revenue. 4) Rebuild the calculated metric correctly: e.g., use 'Revenue' for product revenue and a separate metric for application revenue, then sum if appropriate. 5) Validate against finance data by exporting raw data and reconciling. 6) Document the definitions in the dashboard.
2. What governance and communication steps would you take to prevent future discrepancies and ensure trust in the dashboard?
Read the worked answer
Governance and communication steps: 1) Establish a data governance council including analytics, finance, and marketing to agree on metric definitions. 2) Create a data dictionary that defines each metric, its source, and calculation. 3) Implement data quality checks: regularly reconcile Adobe Analytics revenue with finance systems; set up alerts for anomalies. 4) Use classifications to align revenue categories. 5) In the dashboard, add a disclaimer or tooltip explaining what is included (e.g., 'includes tax and shipping'). 6) Provide a 'data sources' section in the dashboard. 7) Train executives on how to interpret the metrics. 8) Schedule regular reviews of the dashboard metrics with stakeholders. 9) Consider using Adobe's Data Warehouse to create a single source of truth that matches finance.