Utkal Bazaar: The Dashboard That Cheered Too Early
Six-store grocery chain Utkal Bazaar launched a loyalty card in April. By September it had 4,200 members. The quarterly review shows sales up 8%, footfall down 5%, average basket value up 14%, loyalty redemption at 9%, SMS open rate 11%, and 38% of registered mobile numbers failing a validity check. 62% of all sign-ups came from the Bhubaneswar mall store, where a promoter stood at the entrance for six weeks; that store's sales are up 19% while the other five are flat. The marketing head claims the loyalty programme drove the growth and wants ₹6 lakh next quarter for a bigger SMS push. The finance head points out that staple prices rose 12% in the same period, and that many sign-ups were staff and mall employees.
1. Which numbers in this review can be relied on, and what alternative explanations must be ruled out before the loyalty programme is credited with the growth?
Read the worked answer
Trust the transaction data more than the engagement data. Sales, footfall and basket value come from billing and are measured consistently; redemption and open rate are soft and depend on a member base that is partly fake.
Alternative explanations to rule out:
- Store and promoter effect. 62% of sign-ups came from one store with a promoter, and that store is up 19% while the rest are flat. The promoter recruited existing shoppers, so the card is recording customers, not creating them. The growth is localised, which is the opposite of a programme effect.
- Price effect. Staples rose 12%, which accounts for most of the 14% basket increase. Without items per bill, basket value is not a loyalty signal.
- Footfall down, sales up. Fewer, bigger, pricier baskets is not the signature of loyalty; it may signal customers trading down or visiting less.
- Data quality. 38% invalid numbers and an 11% open rate mean the reachable sample is small and biased to the most engaged.
- Staff and mall employees inflate membership without buying.
On this evidence, the 8% cannot be credited to the card.
2. Choose three KPIs for the next review and define each precisely. Explain why each replaces or supplements a number currently in the dashboard.
Read the worked answer
- Repeat purchase rate = customers with two or more bills in the quarter ÷ customers with at least one bill, computed only on validated mobile numbers. Replaces raw membership (4,200) as the headline. Membership counts enrolment; this counts loyalty.
- Incremental basket, read with items per bill = value per bill and units per bill for members versus matched non-members, and for baskets with and without a redeemed offer. Average basket value alone is discredited here because it moves with staple prices.
- Active member rate = members with a purchase in the last 90 days ÷ valid members. With 9% redemption and 38% bad numbers, this will show that most of the base is dormant - the number the marketing head needs before asking for ₹6 lakh.
Guardrails: invalid-number percentage and opt-out rate, tracked monthly. Drop SMS open rate as a headline KPI; at 11% it measures deliverability, not persuasion, and a bigger push mainly increases cost and opt-outs.
3. Design a low-cost test to measure whether the proposed ₹6 lakh SMS push actually changes buying behaviour. State the measure, the comparison and the decision rule.
Read the worked answer
Design: matched-store test with an in-store holdout.
- Pick two stores similar in town size, footfall and basket value that had no mall promoter.
- Randomly assign one as treatment and one as control, and match the SMS lists on recency and past spend.
- Inside the treatment store, hold back 10% of selected members who receive no message. This separates the store effect from the message effect - the mistake made with the mall promoter.
- Send the same offer to the treatment group only, with a unique code per member so redemption is traceable.
Measure: bills per member over six weeks, not revenue, because price movements distort revenue. Secondary: redemption rate and opt-out rate.
Decision rule: the cost per reachable member is ₹6 lakh divided by the valid, contactable base; scale only if incremental contribution from extra bills exceeds that cost with a clear margin. If not, do not spend the money.
Precondition: clean the 38% invalid numbers and require mobile validation at sign-up, or a large share of ₹6 lakh buys nothing at all.