Concept Test With a Split Signal
A consumer hardware startup is validating a smart-lock concept. In a monadic concept test with 400 respondents, purchase intent scored 4.1/5 overall. But the sample splits sharply: renters (n=180) scored 4.6 and homeowners (n=220) scored 3.7. The concept described installation as "no tools, 10 minutes." Follow-up interviews reveal renters love it because they move often and want portable security; homeowners say their doors are non-standard and they doubt the 10-minute claim. The team's go/no-go threshold is a 4.0 mean. Manufacturing tooling requires a $600K commitment in six weeks. The concept also assumes a single SKU fits standard US deadbolts.
1. The overall mean clears the 4.0 threshold. Should the team proceed on that basis? Explain what the split tells you.
Read the worked answer
No - the aggregate mean is the wrong statistic here. A 4.1 built from 4.6 and 3.7 is a mixture, not a population. Proceeding on the mean assumes the concept performs uniformly; the data says it performs very differently by segment.
What the split implies:
- The concept is effectively two products in one test: a strong fit for renters and a weak fit for homeowners.
- The weak segment's objection is diagnostic, not cosmetic: homeowners doubt the installation claim because their doors are non-standard. That is a product/positioning problem (fit and credibility), not a messaging tweak.
- If the addressable market skews homeowner - which is typical for permanent home security - then the segment that likes the concept may be the smaller or lower-value one. The mean hides this.
Before a $600K tooling commitment, I would segment the business case: size the renter opportunity on its own, and treat homeowner fit as an open risk. A threshold applied to a blended mean is a governance failure, not a green light.
2. Recommend how to de-risk the decision in the six weeks before the tooling commitment. Be specific about what you would test and what result would change your mind.
Read the worked answer
The binding risk is physical fit and installation credibility, so test that, not more opinions.
Plan:
- Measure the real door population. Sample doors (rental stock, apartment buildings, homeowner homes) and quantify what share are standard deadbolts. If renters' buildings are mostly standard, the renter segment is viable even if homeowners aren't.
- Prototype the install. Put a rough unit in front of 15–20 non-standard-door owners and time a real installation. If "10 minutes, no tools" fails on real doors, fix the claim or the adapter before tooling.
- Price-test the renter segment. Renters may have lower willingness to pay and a portability requirement; confirm the unit economics on that segment alone.
- Decide the SKU question. Either commit to an adapter/universal-fit design (adds cost and time) or explicitly target standard doors and size the market to that.
What would change my mind toward go: renter segment shows adequate market size and willingness to pay, and a prototype install succeeds on a meaningful share of non-standard doors. What would stop it: renter market too small to carry the tooling, or the install claim collapses under real conditions. Six weeks is enough to learn this; it is not enough to learn it after tooling.