Lakeshore Credit Union: When 62% Said Yes and 0.4% Applied

The survey was not wrong; it was asked the wrong question and the pilot was a weak test. Four things are happening at once:

• Stated versus revealed preference. "Would you definitely use" costs the respondent nothing. Intent questions without a price, a rate, or a repayment term measure enthusiasm, not demand. Nobody was shown an APR or a maximum loan amount.
• Self-selection. A 4.1% response rate means the sample is the members most engaged with the credit union - plausibly older, branch-attached, and least likely to need an instant small-dollar loan. The people who would use it probably did not answer.
• A confounded pilot. Only 2,400 members saw the page, nine applied. Nine is not a demand signal; it is noise. Placement, creative, and audience were never separated from product appeal.
• Different outcomes. "Use" in the survey is not "apply today for this rate." The gap is often price and friction, not interest.

The correct conclusion is that the credit union has not yet tested the product. Killing it on this evidence and scaling it on this evidence are equally unjustified. Say so plainly to the board chair: the survey and the pilot do not contradict each other, they simply are not measuring the same thing.

Spend it on behavioral and pricing evidence, not more intent.

• Discrete-choice / conjoint study (~$20K). 300–400 members from the target segment - not the general member base - trading off rate, loan size, speed, and funding destination. This produces a share-of-preference estimate at specific prices, which is what the product committee actually needs.
• Interviews with the nine applicants and a sample of clickers who did not apply (~$8K). Ten interviews will explain the drop-off faster than any survey: rate, amount, eligibility, identity verification friction.
• Behavioral analysis (~$5K, internal). Mine existing data - small-dollar overdrafts, credit card cash advances, and personal-loan declines - to size the real need. People borrowing elsewhere at 29% reveal more than people answering a survey.
• A properly powered test (~$12K). Run the landing page against a defined audience with a specific offer and a control, long enough to produce a usable denominator.

Sequence the conjoint and the interviews first. If conjoint share at a viable rate is below roughly 5% of the target segment, the recommendation is no - and it will be a documented no. Note the deliverability constraint: six weeks is enough for this if the vendor is booked now.

Frame it as an instrument problem, not a contradiction.

NPS is a lagging, single-number summary collected from a self-selected online panel - the same panel skewed toward engaged, digitally comfortable members who are least exposed to call-center wait times. Call abandonment and complaint volume are operational, census-level, and channel-specific. When a summary survey and operational data disagree, trust the operational data about the specific journey and use the survey only for trend.

Present three things. First, segment the NPS by contact channel; the members who called in the last 90 days almost certainly score far below 41, and that subgroup is the actionable one. Second, connect the metrics: pull call volume per 1,000 members, average speed to answer, and abandonment by hour, and show whether service demand rose because of something marketing did - a campaign, a rate change, a product migration. Third, protect the product decision. Do not let a service-quality problem be read as a product demand signal, and do not let an NPS headline be used to defend a service failure.

The ethical point worth stating: reporting NPS 41 to the board without the abandonment data would be technically accurate and substantively misleading. Marketing owns the narrative that joins them.