Impressions Up 40%, Members Down 160: What the Board Hears
Community Alliance for Housing ran a six-month membership campaign with a goal of 500 new members and a budget of $45,000. Final numbers: 2.1 million impressions, 340 new members, 18 earned media placements, 6,100 email opens, and a post-campaign survey of 400 residents showing unaided awareness rose from 12% to 19% (reported margin of error ±5 points). Landing-page conversion was 2.1% against the 4% the plan assumed. The board chair wants to announce the campaign 'exceeded expectations' because impressions ran 40% above target. The executive director must decide whether to renew the $18,000 paid-social line.
1. Evaluate the campaign against its objectives. Separate outputs, outtakes, outcomes, and impact, and say what the numbers can and cannot support.
Read the worked answer
Outputs: 2.1 million impressions, 18 placements, 6,100 opens. Evidence of activity, not of effect.
Outtakes: awareness moved 12% to 19%, a 7-point gain against a ±5-point margin. That is probably real at 95% confidence, but it is a single post-test with no contemporaneous control - seasonality, unrelated coverage, and the survey's own framing could each explain part of it.
Outcomes: 340 of 500 members, 68% of the primary objective. The campaign missed.
Impact: no data. Nothing here measures retention, dues revenue, or whether the new members are the constituencies the organization needed.
Cannot support: 'exceeded expectations.' The primary objective was missed by 160 members and conversion ran at roughly half the plan's assumption.
Can support: a credible awareness gain and a strong earned-media footprint.
One arithmetic check worth doing before any meeting: 340 members at 2.1% conversion implies roughly 16,000 landing-page visits. The paid social bought volume but not qualified traffic - that is the finding, not the impression count.
2. The board chair plans to announce the campaign exceeded expectations. What do you advise?
Read the worked answer
Advise against it, and hand the chair better language rather than a lecture. The risk is concrete: members, funders, and staff can check 340 against 500. A public overclaim creates a credibility problem far larger than a missed target, and it attaches to the chair, whose credibility is the organization's most valuable asset.
Offer this framing instead: 'We grew awareness seven points and added 340 members - short of our 500 goal. We know why: conversion ran about half of what we projected. Here is what we are changing.' Boards absorb a diagnosis better than a defense.
Support it with a one-page dashboard: the four numbers, the baseline, and the two decisions that follow - paid-social renewal and conversion work on the landing page. If the chair needs a success line, scope it accurately: 'exceeded our reach targets' is true and clearly bounded. Never let 'impressions above target' stand in for 'campaign succeeded'; that is the exact substitution the board will repeat next year.
3. Design the measurement approach that would actually answer whether to renew the $18,000 paid-social line.
Read the worked answer
The question is marginal, not total: would another $18,000 in paid social return more members than the next-best use of that money?
- Holdout design. Withhold paid social from a matched set of geographies or audience segments for the full test period and compare member acquisition per dollar against exposed segments. Without a holdout you are measuring the economy, not the channel.
- Right unit of cost. Track cost per acquired member, not cost per click or per impression. The all-in figure here is about $53 per member across the whole budget; isolate paid social's share before deciding.
- Instrument the path. Tag the landing page so every conversion carries a source, and run two or three offer or message tests rather than one.
- Pre-commit to a rule. Renew if paid social's incremental cost per member is at or below the blended target and the holdout shows a measurable lift. Cut or reallocate if the holdout shows no difference.
Name the power problem out loud: with roughly 340 conversions you can detect only large effects, so run a full quarter and register the threshold before you start, not after you see the result.