Raising a new action's arbitration weight to hit a 15% impression target at Tailspin Telecom
Tailspin Telecom has five retention actions. Arbitration uses propensity multiplied by value, with weights: save-offer 1.0, device-upgrade 0.8, plan-review 0.6, and a new flex-bundle action 0.3. Flex-bundle has a propensity of 0.18; save-offer has 0.40. The business wants flex-bundle to receive at least 15% of impressions during a 30-day test. Contact policy limits customers to two offers per week, and a suppression rule blocks customers with overdue bills. Flex-bundle is eligible only for customers on a month-to-month contract, about 22% of the base. Analysts propose raising flex-bundle's weight to 5.0. The contact-center team worries that higher-weight actions will crowd out service messages. No simulation has been run. The test budget covers all channels.
1. Calculate the approximate arbitration scores before and after the proposed weight change for flex-bundle and save-offer, and explain why the 15% impression goal may still fail.
Read the worked answer
Before: flex-bundle 0.18 x 0.3 = 0.054; save-offer 0.40 x 1.0 = 0.40, so flex-bundle loses. After: flex-bundle 0.18 x 5.0 = 0.90; save-offer remains 0.40, so flex-bundle wins whenever both are eligible and no other action scores higher. But the 15% goal is a share of impressions, not a share of auctions. Flex-bundle is eligible only for month-to-month customers, about 22% of the base, and contact policy plus overdue-bill suppression remove many impressions. If flex-bundle wins most of its eligible auctions but is eligible in only a fraction of all interactions, its overall impression share can still be below 15%. The denominator matters: 15% of all impressions is a different target from 15% of eligible month-to-month impressions. Weight cannot create eligibility.
2. What are the risks of raising the weight to 5.0, and what controlled alternative would you propose?
Read the worked answer
Raising the weight to 5.0 can starve device-upgrade and plan-review, saturate the two-offer contact limit, and distort adaptive learning because flex-bundle receives far more responses while other actions receive fewer. It also creates a weight arms race: other owners will ask to raise their weights, and the relative ranking becomes meaningless. A controlled alternative is a champion/challenger arbitration strategy with a flex-bundle boost applied only to a random test group, or an exploration allocation that gives flex-bundle a small random share among eligible customers. Run simulation first to estimate share and contact-policy exhaustion, then compare the test group to a holdout on incremental retention and value. Guardrails should include complaints, contact-policy exhaustion, and service-message crowding.
3. How should Tailspin measure whether flex-bundle deserves a permanent place?
Read the worked answer
Define success before the test: incremental saves or value per eligible customer, not raw flex-bundle impressions. Randomize at customer level into current arbitration, flex-boosted arbitration, and a holdout; run long enough to cover the contact-policy cycle and to accumulate responses for the flex-bundle adaptive model. Track acceptance, downstream conversion, save rate, complaints, contact-policy exhaustion, and the share of service messages. Segment by contract type because flex-bundle eligibility is concentrated there. If the adaptive model for flex-bundle starts with little data, its propensity is noisy; the test should allow learning and avoid peeking at early results. A permanent place is justified only if the boost produces incremental value without unacceptable customer or contact-policy cost.