Lumen Software: a churn model, a data broker and a discount-happy sales team
Lumen Software sells subscriptions to mid-market customers. The chief revenue officer has asked the finance business partner to build an account-level churn risk score for the CRM by next quarter, with a £120k budget. Three data sources are available: product usage logs in the data warehouse, free-text CRM notes written by sales reps, and a purchased external file of 180,000 contact and firmographic records bought from a data broker. Legal has found that the broker contract contains no processing terms and no statement of how consent was obtained. Usage logs exist only for the platform launched in 2022, which covers 66% of the customer base; older accounts have no telemetry. Region codes are missing on 12% of accounts. The CRO wants the score used to prioritise which accounts receive retention discounts.
1. Assess the suitability of the three data sources. What should the business partner do about the purchased dataset and the missing usage history?
Read the worked answer
Source-by-source assessment
- Usage logs (internal). Good accuracy and timeliness, but only 66% coverage. The 34% without telemetry are older accounts - plausibly the most likely to churn and the least scorable. A model trained only on new accounts will systematically understate risk in the segment that matters most.
- CRM notes (internal). Free text written by reps with different habits and incentives. Low consistency and validity; usable only after a tagging exercise, and biased by how often each rep logs. Valuable as a supplement, dangerous as a primary input.
- Purchased file (external). Two problems, one fatal. The commercial problem is that 180,000 contacts is far more than Lumen needs - a data minimisation failure. The legal problem is that there is no processing agreement and no demonstrated lawful basis, which means ingesting it into a model is likely unlawful processing and would fail any data protection impact assessment or audit.
Recommendation
- Do not load the purchased file. Ask legal to obtain a compliant basis or a replacement source with contractual terms; if that fails, drop it. Run a DPIA before any modelling and document the lawful basis and retention.
- Build v1 on internal data only, scoped explicitly to the 66% with telemetry, and give legacy accounts a separate rule-based treatment (contract end date, support ticket trend, payment behaviour) with a visible 'insufficient data' flag.
- Do not impute the missing 12% region silently. Keep an 'unknown' category; silent imputation hides a reporting weakness that governance should fix.
- State coverage in every output. A score that quietly excludes a third of the base will be trusted more than it deserves.
2. How should the finance business partner respond to the CRO's request? Address value, feasibility, and the risk that the score is used to justify blanket discounting.
Read the worked answer
Start with the decision, not the model
Ask what decision the score changes and what it is worth. If the alternative is a simple rule - contracts expiring within 90 days plus falling logins - a model must beat that rule to justify £120k. Quantify the discount cost: if retention discounts average 8% of annual recurring revenue, the score has to protect more revenue than it gives away, on average, or the programme destroys margin whatever the model's accuracy.
Be honest about feasibility
A production-grade score for the whole base in one quarter is not deliverable. Offer a staged commitment: in 90 days, an agreed churn definition, cohort analysis, usage-based risk tiers for the 66% with telemetry, and a simple rule for the rest; in 12 months, a validated model with lawful data, monitored for drift.
Name the behavioural risk explicitly
The CRO's stated use - ranking accounts for discount - creates two problems. First, discounting every flagged account rewards the model for finding risk and the sales team for spending margin on it. A flagged account may need a service intervention, an executive sponsor call or a pricing review, not a discount. Second, if treatment is applied only to high scorers, Lumen never learns whether the score works; the programme needs a control group and pre-agreed success measures (revenue retained per pound of discount, churn rate by tier).
Governance
Agree once, in writing: who owns the model, what it may not be used for (individual employment or credit decisions), how often it is revalidated, and who signs off changes. Present limitations to the CRO in business terms - coverage, no causal claim, and the discount-cost test - before any build starts.