The Startup's Churn Announcement
A software company's quarterly update to investors states: "Customer churn improved to 2.1 percent this quarter, our best result in two years, showing that our new onboarding program is working." Context available to an analyst: the company changed its churn definition this quarter, moving from "accounts that canceled" to "accounts that canceled and did not reactivate within 30 days." It also raised prices 12 percent at the start of the quarter and discontinued its cheapest plan tier. The onboarding program launched mid-quarter. Last quarter's reported churn under the old definition was 3.4 percent. The company has not restated prior quarters under the new definition.
1. What does the announcement imply about causation, and what does the available context suggest instead?
Read the worked answer
Implied causation: The update asserts that the onboarding program is working - i.e., that the drop to 2.1 percent was caused by onboarding.
What the context suggests: At least three other changes occurred in the same quarter, any of which could move the number:
- Definition change. Moving from "canceled" to "canceled and not reactivated within 30 days" mechanically lowers measured churn: some cancelers now count as retained if they return within a month. The 2.1 percent is not comparable to the prior 3.4 percent.
- Price increase. A 12 percent price rise would typically increase churn, not lower it - so if churn fell, something else may be offsetting it, or the definitional effect is large enough to swamp the price effect.
- Tier discontinuation. Dropping the cheapest plan pushes price-sensitive customers off the platform or upmarket; either way it changes who remains in the denominator, and it may remove the customers most likely to churn.
The honest reading: the quarter contains a measurement change, a price shock, a product-mix change, and the program - all at once. Attributing the result to onboarding is unsupported. A careful reader notes that "best result in two years" is especially misleading because the two years were measured under a different definition.
2. The company says the onboarding program "is working." What would a reader need in order to evaluate that claim? List the specific items.
Read the worked answer
To evaluate the causal claim, a reader would need:
- Restated history. Prior quarters recalculated under the new 30-day-reactivation definition, so the 2.1 percent can be compared to a like-for-like baseline.
- A decomposition. How much of the change is definitional, how much price-driven, how much from the tier discontinuation, and how much residual could plausibly be onboarding.
- A comparison group or staged rollout. Churn among customers who went through onboarding versus those who did not, ideally with onboarding assigned in a way that isn't correlated with customer type.
- Timing detail. The program launched mid-quarter; if it affects churn, the effect should be partial in this quarter and larger next quarter. A full-quarter effect appearing immediately would be suspicious.
- Segment breakdown. Churn by plan tier and by tenure, to see whether the drop is concentrated where onboarding would plausibly act.
- Absolute counts, not just rates. A percentage can fall while absolute cancellations rise if the base shrank - relevant here, since a tier was discontinued.
A strong answer distinguishes restatement (needed just to compare), decomposition (needed to apportion causes), and design (needed to isolate the program).
3. Draft one sentence the company could honestly write in place of the original, preserving what the data supports.
Read the worked answer
A defensible replacement: "Reported churn was 2.1 percent this quarter under our revised definition, which counts accounts as churned only if they do not reactivate within 30 days; because this definition and several pricing changes took effect this quarter, the figure is not directly comparable to prior periods, and we are still assessing the onboarding program's effect."
Why this works: it reports the number, discloses the definition change, flags the confounds (pricing changes), and withdraws the causal claim without denying the result. It keeps the reader oriented while being accurate. The original failed not because the number was wrong but because it was placed in a causal sentence it could not support.