Measuring and Analyzing Product and Business Performance
Metric hierarchy
Know the difference between product metrics (activation, engagement, feature adoption, retention, churn) and business metrics (revenue, gross margin, CAC, LTV, payback, share). North-star metrics should reflect delivered customer value and ladder to revenue. Traps: celebrating vanity metrics (page views, downloads) and confusing correlation with causation.
Cohorts and retention
Cohort analysis reveals whether retention improves over time; blended averages hide it. Churn should be split into voluntary/involuntary and by segment. A trap is treating a rising average as improvement when the mix shifted (Simpson's paradox).
Attribution and experiment design
Attribution models (first-touch, last-touch, multi-touch) each bias results; know their limitations. Prefer controlled experiments where feasible. Watch for questions that accept a single-touch attribution number as ground truth.
Diagnosing underperformance
Use a structured approach: is it awareness, conversion, retention, or pricing? Compare against targets and benchmarks, segment the data, and form hypotheses. Traps include blaming marketing for a product-quality-driven churn issue, or blaming sales for a lead-quality problem.
Rule of thumb
For every metric ask: compared to what, over what period, for which segment? A number without a baseline and a segment is not analysis.
Worked example
Revenue is flat but churn rose and new sales rose equally. Blended revenue hides deterioration - cohort retention shows the leak. Fix retention before scaling acquisition spend.