Real-time next-best-action versus nightly batch segmentation at Northwind Bank
Northwind Bank runs outbound email campaigns from a nightly data warehouse. Each night a batch job segments 4.2 million customers into 12 segments and sends the same mortgage offer to everyone in a segment. The bank now wants real-time inbound next-best-action on its mobile app and call-center desktop: 30 eligible actions across deposits, cards, and loans; three channels; a 200 ms response budget; and a compliance rule that no customer may see more than one lending offer per week. Customer data arrives from three systems: core banking (daily batch), card transactions (streaming with 5-minute latency), and web clickstream (real-time). Marketing asks whether they can keep the nightly segmentation and simply add a real-time container at the end. The compliance team also requires an auditable reason whenever a lending offer is shown.
1. Should Northwind keep the nightly segmentation and add a real-time container at the end, or redesign around next-best-action? Explain the decision using latency, per-customer context, and the one-lending-offer-per-week rule.
Read the worked answer
Northwind should not treat the real-time container as a bolt-on to nightly segmentation. The batch job selects one offer per segment, so by the time the container runs, the candidate set is already collapsed and does not reflect the customer's current card transactions or web session. A 200 ms budget also rules out synchronous calls to core banking; the profile must be assembled ahead of the decision from a daily batch data set for core banking and real-time data flows for card and clickstream events. The one-lending-offer-per-week rule is the decisive constraint: it is a cross-channel contact policy that must be evaluated at interaction time against interaction history. If the batch sends a mortgage email on Monday, the mobile app must know on Tuesday that the lending slot is used. A nightly process cannot enforce that. The right design is NBA in a real-time container, with batch feeds shaping the profile and real-time events shaping context, and contact policy plus suppression evaluated in NBA Designer before arbitration. Audit reasons should be written at decision time, not reconstructed later from batch logs.
2. What should Northwind instrument to prove the inbound NBA is better than the batch campaign, and what comparison would be misleading?
Read the worked answer
Instrument the full decision, not just the click: customer id, channel, timestamp, issue/group/action, treatment, propensity, arbitration score and rank, eligibility/applicability/suitability result, contact-policy and suppression reason, impression, accept/decline, and downstream conversion. Write these to interaction history. To prove uplift, randomize at customer level into a control that keeps the old batch experience and a test that receives inbound NBA, or run a champion/challenger with a holdout. Compare value per eligible customer and contact-policy exhaustion, not raw click-through. A misleading comparison would be inbound app acceptance versus email acceptance, because channel intent and audience differ; another is a before/after comparison across a seasonal period. Also avoid counting an impression as success when the customer was ineligible for the control. If simulation is used, treat it as a pre-check on strategy logic, not as evidence of causal lift.