Planning and Strategy
What to know
Planning questions test whether you can turn a business goal into a measurable, testable structure before touching the UI. Candidates must understand success metrics (primary metric vs. supporting metrics), KPI definition, the difference between a business goal and a test hypothesis, and how activity types (A/B test, multivariate test, experience targeting, automated personalization, recommendations) map to different objectives. Also know traffic estimation (how much traffic an experience needs to reach statistical confidence), minimum sample size reasoning, and the Target setup prerequisites: mbox/at.js or Web SDK implementation, the global mbox, and audience readiness.
Traps in questions
- A question describes a vague goal ("increase engagement") and asks the first step - the answer is defining a measurable success metric, not launching a test.
- Confusing a hypothesis with a goal. A hypothesis is a testable statement ("adding social proof increases conversion") tied to a metric.
- Choosing a test type that cannot answer the question. If you need to test many elements and interactions, that is MVT, not A/B.
- Ignoring sample-size feasibility: an activity with 6 experiences and low traffic will not reach significance quickly.
Worked example / rule of thumb
Goal: "Increase newsletter signups on the homepage." Primary metric = signup conversion rate; supporting metrics = click-through on CTA and bounce. Hypothesis: "Adding a value statement above the form increases signups." Test type: A/B with two experiences (control vs. variant). Rule of thumb: one primary metric per test; supporting metrics inform but do not decide the winner. If you cannot name the metric and the decision rule before launch, the plan is not ready.