Measures, Churn Prediction, and Outbound Export for a Subscription Box Service
Adventure Works Box has 500,000 subscribers. Leadership wants a monthly churn prediction, a customer lifetime value measure, and a nightly export of at-risk subscribers to an email platform. The data science team has historical cancellation labels for 18 months. The email platform accepts batches of up to 100,000 records per call and rate-limits to 10 calls per minute. The export must complete within a two-hour nightly window, and personally identifiable information must be minimized in the outbound payload. The team is unsure whether to build the prediction first or the export first, and whether the lifetime value measure should use predicted or historical revenue.
1. What build order do you recommend among the measure, the prediction, and the export, and why?
Read the worked answer
Build in dependency order: measure, then prediction, then export.
The lifetime value measure depends only on unified profiles and transaction history, so it can be built and validated first, giving the team a working measure pipeline and a validated data foundation. It also produces an input the churn model can use as a feature.
The churn prediction depends on that same historical data plus labels, and its output (a churn score or risk tier) is what the export consumes. Building the export before the prediction exists means exporting a placeholder, which wastes effort and risks the email platform being wired to the wrong field.
The export comes last because it depends on both: it needs the risk tier from the prediction and possibly a value field from the measure to prioritize the audience. Building it last also means the payload schema is designed against real output rather than assumptions.
Sequence the work with validation gates: confirm the measure reconciles to finance figures, confirm the model's scores are calibrated on a holdout period, then wire the export.
2. The export must fit a two-hour window under a 100,000-record batch limit and 10 calls per minute. How do you design it, and what happens if the at-risk audience exceeds capacity?
Read the worked answer
Compute capacity first. At 10 calls per minute, two hours allows 1,200 calls; at 100,000 records per call, theoretical capacity is 120 million records - far above the 500,000 subscriber base. So the rate limit is not the binding constraint; the real risks are payload size, retry behavior, and whether the audience is filtered sensibly.
Design:
- Filter before export. Export only subscribers above the risk threshold and eligible for contact (consented, not suppressed), which keeps batches small and improves relevance.
- Batch and throttle deliberately. Send in batches sized below the limit with pacing that respects the 10-per-minute cap, and implement retry with backoff for failed batches so a transient error does not restart the whole export.
- Make it idempotent. Include a run identifier so a retried batch does not create duplicate sends in the email platform.
- Minimize PII. Send only the identifiers and attributes the email platform needs - a hashed or platform-side customer key, risk tier, and merge fields - rather than names, full emails, and addresses in bulk.
If the audience exceeds capacity, prioritize by risk score and value, and export the top tranche with a documented cutoff rather than truncating arbitrarily.
3. Should the lifetime value measure use predicted or historical revenue? Advise the team.
Read the worked answer
Use historical revenue as the primary measure and treat predicted revenue as a separate, clearly labeled measure.
Historical lifetime value is auditable: it reconciles to transaction records, so finance and marketing can trust it and use it for reporting and segmentation. It is also stable - the same customer yields the same value until new transactions occur, which makes downstream segments reproducible.
Predicted lifetime value is forward-looking and useful for prioritization, but it carries model error and will shift whenever the model is retrained, causing segments and exports to change for reasons unrelated to customer behavior. Mixing the two into one measure makes it impossible to tell whether a change reflects the customer or the model.
Recommendation: build historical lifetime value first as the governed measure, then add a predicted value measure once the churn model and its features are validated. Label them distinctly, document the model version behind the predicted measure, and if the business wants a blended view, present the two side by side rather than averaging them into a single opaque number.