Sign-up form test without deeper discounts
An online skincare brand has 300,000 monthly site sessions, and 78% are mobile. Its only sign-up form is a full-screen popup that appears 5 seconds after page load on every page, including checkout and the blog. The offer is '10% off your first order.' Last month the form had 180,000 views, 2,160 submits (1.2%), and 1,450 new profiles after double opt-in. The welcome flow has three emails and converts 4.2% of subscribers within 14 days, but the team suspects the discount attracts bargain hunters: 38% of welcome-flow buyers use the code and then never purchase again. The brand wants to lift form submissions without increasing discount depth and without hurting deliverability. It can change targeting, timing, copy, and the incentive, but it cannot offer free shipping because margins are tight.
1. Design a single-variable A/B test for the sign-up form. State the hypothesis, the variant, the success metric, and one guardrail metric.
Read the worked answer
Test one variable: form trigger. Hypothesis: an exit-intent or delayed form on product pages will raise submit rate without increasing discount-only signups. Control: current 5-second full-screen popup on all pages. Variant: mobile slide-in shown on product pages after 20 seconds or exit intent. Success metric: submit rate per form view after double opt-in. Guardrail: 30-day unsubscribe or spam-complaint rate and 60-day revenue per new subscriber. Randomly assign visitors, run for at least two weeks or until each arm has enough submits for a meaningful difference, and do not change creative or offer mid-test. This isolates timing/targeting from the incentive.
2. How would you target the form to reduce discount-only signups while protecting list growth?
Read the worked answer
Suppress the form for existing customers, profiles already subscribed, and checkout pages. Target new visitors on product and collection pages, where intent is higher, and use a less aggressive mobile slide-in instead of a full-screen takeover. For blog traffic, offer a routine guide rather than the 10% code. Keep double opt-in and clear consent language. This reduces bargain hunters because the incentive is not shown to people who are already customers or just browsing content, and it protects the checkout experience.
3. The welcome flow conversion is 4.2%, but repeat purchase is weak. What changes would you make to the flow, and how would you measure whether they worked?
Read the worked answer
Split the welcome flow after the first purchase. Purchasers get onboarding, usage education, and a replenishment reminder without another discount; non-purchasers get social proof, reviews, and a single urgency message. Measure 60/90-day repeat purchase rate, revenue per recipient, AOV, and discount redemption by split. Do not judge by welcome conversion alone, because a 4.2% conversion can hide low repeat value. If the purchaser branch lifts repeat rate without deeper discounts, keep it; if not, test a replenishment timing change rather than adding more offers.