Analytics, Reporting, and Performance Measurement
Measure what the decision needs
Analytics questions test KPI definitions and causal discipline. Define each metric precisely: revenue gross versus net of returns and discounts; sessions versus users; conversion rate = orders / sessions (state the denominator); AOV = revenue / orders; units per transaction; margin; new versus returning split; repeat purchase rate; churn; and the funnel from session to PDP view to add-to-cart to checkout to order.
Tooling: an event-based model records events with parameters, and e-commerce reporting depends on correct, deduplicated purchase events. Attribution (last-click, first-click, linear, data-driven) assigns credit; it does not establish causation. Incrementality needs experiments - holdouts, geo tests, or A/B tests with a pre-registered sample size and duration.
Traps: calling a dashboard a source of truth without definition control and an owner; peeking at A/B results and shipping on the first significant day; confounding seasonality with campaign effect; comparing conversion rates across segments of different intent; reporting revenue that includes canceled or returned orders.
Rule of thumb: every KPI needs a name, a formula, a source system, an owner, and a refresh cadence. If any is missing, the number will drift.
Worked example: a test shows +8% relative lift in conversion for variant B, on 4,000 sessions per arm against a 3% baseline. The standard error of the difference is roughly 0.4 percentage points, so a 0.24-point absolute lift is indistinguishable from noise - keep running or accept the null rather than shipping on day three.