Weekly Cancellation Rate: Variable Limits and Phase Change
A subscription business monitors weekly cancellation rate among new subscribers. Weekly eligible base varies from 800 to 2,400 because marketing campaigns are uneven. The quality team currently uses a p-chart with control limits calculated from the average sample size (n = 1,500), giving UCL = 4.6% and LCL = 1.4%. In week 12, cancellations are 43 of 900 (4.8%), above the UCL, and the team launches a retention campaign. In week 13, cancellations are 99 of 2,300 (4.3%), inside the constant limits, so no action is taken. A new onboarding flow went live at the start of week 10. The team also notices that weekly rates are more variable than the p-chart predicts, even after the onboarding change. The data are counts of cancellations per weekly cohort, with each subscriber counted once. Marketing wants a single dashboard number; operations wants weekly signals. The belt has 52 weeks of historical data and can recover exact denominators.
1. Evaluate the chart choice and the two signals. What should the team do differently?
Read the worked answer
The constant-limit p-chart is wrong for these data because the weekly denominator varies from 800 to 2,400. With n = 900, the standard error of the proportion is larger than with n = 2,300; using limits based on n = 1,500 makes small weeks look more extreme and large weeks look less extreme. Recalculate a p-chart with variable limits: UCL/LCL = p-bar +/- 3*sqrt(p-bar(1-p-bar)/n_i). Week 12 may no longer be a signal, and week 13 may become one. Also split the chart at week 10, when the onboarding flow changed; limits from the pre-change process should not be applied to the post-change process. The extra week-to-week variability beyond binomial expectation suggests overdispersion; if it persists after the phase split, use a Laney p' chart or an individuals chart on weekly rates with an appropriate transformation. Do not launch a retention campaign from a misread point.
2. How should the team handle the onboarding change and the desire for a single dashboard number?
Read the worked answer
Treat the onboarding change as a new phase. Establish a pre-week-10 baseline with its own limits, then a post-change phase with new limits after enough weeks. Compare the phases using a two-proportion test or a logistic regression that adjusts for cohort mix and seasonality, not by eyeballing one week. For marketing's single dashboard number, show the overall cancellation rate with a confidence interval and a note on mix; for operations, keep the weekly control chart with exact denominators. A funnel plot is another way to show which weeks are unusual given their sample size. If the process is overdispersed, the dashboard should show the control limits from the Laney chart, not a misleadingly narrow binomial interval. The single number and the weekly signal serve different decisions; collapsing them loses the ability to detect real change.
3. What monitoring plan and sample-size considerations would you recommend going forward?
Read the worked answer
Monitor weekly with exact counts and variable limits, and use a phase-based chart after any known change. Decide the minimum detectable shift and the minimum weekly n; if marketing produces weeks below that n, report the rate with a wide interval or combine weeks, but do not pretend the rate is as precise as a large week. Because subscribers are counted once in their cohort, the p-chart's independence assumption is reasonable, but cohort mix can still create overdispersion; check with a Laney chart and, if needed, risk-adjust by acquisition channel or plan type. Set a review cadence: operations reviews weekly signals, marketing reviews monthly trends. Add leading indicators such as first-week engagement or support contacts. Do not over-adjust to single points; require a sustained pattern or a point beyond variable limits, and confirm with the next cohort before major action.