Northline Retail: Churn Prediction Ambition vs Data Readiness
Northline Retail operates 212 stores and an e-commerce site. Its analytics team produces weekly sales reports from the point-of-sale system: 4.2 TB of structured transaction records containing store ID, SKU, timestamp, and amount. The customer service director has recorded 340 hours of inbound calls and wants an AI system to predict which callers will churn. The CIO has asked for one data analytics roadmap and wants to know whether the call recordings and transaction data belong in the same project. The team has not transcribed the calls, most call records carry no customer identifier, and there are no historical churn labels. A vendor offers a sentiment model that returns a score between 0 and 1. The CFO expects a predictive churn model in eight weeks, and the customer service director has already told the board that 'sentiment will tell us who is leaving.' The privacy team has not reviewed the recordings.
1. Classify the two data sources and identify which analytics types (descriptive, diagnostic, predictive, prescriptive) the organisation can realistically perform today. Justify each classification.
Read the worked answer
POS transactions - structured, quantitative. Rows and columns are fixed; each field has a defined type. This supports descriptive analytics (weekly sales, basket size) and diagnostic analytics (why a store's sales fell). It also contains the raw material for predictive analytics only if a labelled outcome (churn) can be attached.
Call recordings - unstructured, qualitative. Audio has no predefined schema. Before analysis it needs transcription, speaker separation, and entity resolution to link calls to customers. Until that happens the data supports manual quality review at best, not churn prediction.
Sentiment score - derived, not a label. A score between 0 and 1 measures expressed tone in a call. Churn is a behavioural outcome: a customer stops buying or cancels. Sentiment may be a feature, but using it as the label would train the model to predict mood, not departure. The organisation today can do descriptive reporting and some diagnostic analysis; predictive churn is not yet supportable, and prescriptive analytics is premature.
2. Advise the CIO on sequencing the roadmap and on the eight-week predictive churn commitment. What must be true before a churn model can be validated?
Read the worked answer
Sequence the roadmap by data readiness, not by executive enthusiasm. First, build a trusted descriptive baseline: define churn operationally (no purchase for 90 days? no login? cancelled contract?), establish the observation window, and link transactions to a stable customer key. Second, run diagnostic analysis on the transaction data - cohort retention, recency/frequency, service-contact patterns - because it is structured and already governed. Third, invest in the call data: consent and privacy review, transcription, and identity matching. Only then test whether call features add predictive value beyond transaction features.
The eight-week promise is not credible for a validated churn model. A defensible model needs historical labels covering a full outcome window, a feature window that precedes the outcome to avoid leakage, and a holdout or time-based validation set. With no labels today, eight weeks may allow a pilot on transaction data, but not a production claim. Tell the CFO what can be delivered in eight weeks (baseline, label definition, feasibility) and what cannot.
3. The vendor's sentiment score is proposed as the churn label. Explain why this is or is not acceptable and what governance issue arises from using the call recordings.
Read the worked answer
Using sentiment as the churn label is not acceptable. It creates label leakage and construct mismatch: the model would learn to reproduce a sentiment classifier's output, and a positive sentiment score does not mean a customer renewed. Validation against actual churn would then fail or, worse, appear successful if sentiment happens to correlate with complaints. The label must be an observed behavioural outcome with a defined time window.
The call recordings raise a separate governance issue. Recording customers without privacy review creates risk under consent, purpose-limitation, and retention rules. Using the audio for model training is a new purpose that may not be covered by the original notice. Before transcription, the team needs a lawful basis, a retention schedule, controls on sensitive content that customers may disclose, and a process for handling data-subject requests. The vendor score also needs documentation of training data and known bias. The board should hear that sentiment is a possible feature after governance clearance, not a churn label.