The loyalty scheme that flatters the wrong customers
A mid-market garden-centre retailer has 140,000 loyalty members. Management reports that members spend 38% more per visit than non-members and proposes cutting the 2% discount to fund a £400,000 loyalty app. A junior analyst notes that membership is free at the till and is offered most enthusiastically to customers already spending over £60. Cardholders also receive a birthday voucher worth £10 that is redeemed by 61% of recipients, mostly on purchases under £25. Redemption of the 2% discount is near-universal. The finance director wants a decision within three weeks; the marketing director believes the scheme "proves" loyalty works. No control group has ever been tracked, and the app supplier's business case assumes member spend uplift persists at 38% after launch.
1. Assess whether the 38% spend premium is credible evidence of the loyalty scheme's effectiveness. What data would you require before the board commits to the app?
Read the worked answer
The 38% figure is a correlation, not a causal effect. Membership is free and offered at the till, most eagerly to high-spending customers, so the scheme is partly selecting big spenders rather than creating them. This is self-selection bias, and it inflates the apparent uplift.
Three further distortions compound it:
- Regression to the mean is not the issue here; selection is. But the birthday voucher's 61% redemption on baskets under £25 is almost pure subsidy - it rewards a purchase that may have happened anyway, and it trains members to wait for the voucher.
- The 2% discount is near-universally redeemed, so it behaves as a price cut, not a loyalty device. Its cost should be judged on incremental margin, not on member satisfaction scores.
- No control group means there is no counterfactual. The supplier's business case assumes the 38% persists post-launch - an assumption with no evidential basis.
Before committing, I would require:
- A matched control design: compare members with similar prior spend, frequency and basket size to non-members, or run a holdout cohort denied membership for 6–12 months.
- Incremental margin per member, net of discount, voucher and app cost, not gross spend.
- Cohort retention curves - are members retained longer, or just spending more while they were always going to?
- Voucher cannibalisation analysis: what proportion of birthday redemptions were unplanned visits?
- The supplier's assumption sensitivity: what happens to payback if uplift falls to 10%?
Recommendation: approve a pilot with a holdout, not the full £400,000.
2. The marketing director argues the scheme builds emotional loyalty. How would you test that claim, and what would change your mind?
Read the worked answer
Emotional loyalty should show up as behaviour that survives the removal of the incentive. That is testable.
Tests I would run:
- Discount withdrawal test: a small cohort receives the app and birthday voucher but not the 2% discount. If retention holds, loyalty is emotional; if it collapses, it was transactional.
- Attitudinal measures: Net Promoter Score, stated preference and a "would you still shop here if the scheme ended?" question, segmented by spend decile. Stated intent is weak evidence alone, so triangulate with behaviour.
- Share of category: does membership increase the proportion of a customer's total garden spend captured, or merely the frequency of small top-up visits? Share of wallet is a better loyalty signal than visit count.
- Voucher response by tenure: if long-standing members redeem as heavily as new ones, the voucher is a habit, not a relationship.
- Referral and non-incentivised advocacy: do members bring others without a reward?
What would change my mind: if a holdout comparison showed members retained 15+ percentage points better after 12 months and the no-discount cohort held its retention, I would accept a genuine loyalty effect. If retention differences vanish once prior spend is controlled, the emotional-loyalty claim is unsupported and the app case must rest on operational benefits (data capture, personalisation) rather than loyalty uplift.
I would also flag the measurement trap: satisfaction surveys will look good regardless, because members chose to join. The board should weight behavioural evidence above attitudinal scores.