A health score that flags the wrong accounts
Halcyon Workforce is a B2B scheduling platform with 600 accounts, 14 CSMs, and a HubSpot health score made of five equally weighted inputs: login count, number of support tickets, average CSAT, NPS, and contract value. Accounts below 50 are green, mid-band accounts are amber, and the lowest band enters a save playbook. The score flags 38% of accounts red, so CSMs largely ignore it. Several high-value accounts with many low-severity password tickets are red, while accounts with no tickets and falling schedule creation are green. Last quarter churn was 9%; 60% of churned accounts were green 90 days before churn. NPS response rate is 11%. Support and CS use different definitions of active account. The board wants a churn-risk report next month.
1. Critique the current health score. Which inputs are misleading, and why?
Read the worked answer
The score mixes risk and value and uses too many sparse or ambiguous inputs. Ticket count is not a health signal by itself: password-reset tickets can mean active usage and a responsive support relationship, while silence can mean disengagement. CSAT and NPS are sparse; an 11% NPS response rate cannot carry 20% of the score. Contract value is a prioritization input, not a health input; it makes high-value accounts look unhealthy or healthy for the wrong reason. Login count is shallow and can be inflated by a single admin. There is no usage depth, no trend, no recency, no segment logic, and no handling for missing data. The churn evidence is the strongest test: 60% of churned accounts were green 90 days out, so the score is not predictive. CS and support also disagree on active account, so the inputs are not comparable.
2. Redesign the score and the operating model around it. What would you include, how would you weight it, and how would CSMs use it?
Read the worked answer
Separate value from risk. Value tier (contract size, expansion potential, strategic flag) decides coverage and playbook intensity. Risk score predicts renewal or expansion risk. Build risk from leading indicators with recency and trend: depth of product use (schedules created, active admins, seats active), week-over-week trend, onboarding milestone completion, unresolved high-priority tickets, reopened tickets, CSAT trend, invoice or payment issues, and executive engagement. Use survey data only as a low-weight signal with missing-data handling. Weight by segment, because a 50-seat account and a 5,000-seat account have different usage baselines. Refresh weekly, show the reason codes, and set bands with a review queue rather than a single red threshold. Validate against the last four quarters of churn and expansion; if precision is poor, change weights before adding more inputs. CSMs should work the reason code, not the number: for example, usage decline + no admin login triggers an adoption play, while unresolved P1 + low CSAT trend triggers a service recovery play.
3. What would you put in the board churn-risk report, and what caveats would you state?
Read the worked answer
Board report:
- Cohort retention and gross churn by value tier, not one blended rate.
- Health-band migration: green-to-red, red-to-saved, and red-to-churned.
- Leading indicator trends for the top risk reasons: usage depth, admin activity, unresolved high-priority tickets, reopen rate, CSAT trend.
- Playbook completion and save rate, with the definition of
saved. - Churn reasons from closed-lost interviews, tagged to support, product, commercial, or customer action.
Caveats: the score is a prioritization tool, not proof of causation. NPS coverage is 11%, so survey results are directional. Support may influence retention but does not control product fit or commercial terms. Compare matched cohorts and state the sample size. Do not present a single green health score as a forecast; show calibration and the accounts where the model was wrong.