Restructuring a tangled inbox before enabling Fin at Kestrel Health
Kestrel Health runs three product brands from a single Intercom workspace. Support has never been properly configured: 14,000 conversations sit unattributed, 60 overlapping assignment rules were built by different team leads over three years, and agents rely on personal saved views. There are no close reasons and tags are inconsistent - 'refund', 'Refunds', and 'billing-issue' all exist. Nine languages arrive through chat and email; only English and Spanish have dedicated agents.
Fin was enabled on one brand six weeks ago. It answers well, but handoffs land in a general unassigned queue: 40% of Fin handoffs wait more than 24 hours for a first human reply, and customers reopen the conversation asking whether anyone is there. A migration to a structured inbox is scheduled for next month. The support director wants to 'clean the 14,000 conversations first' before changing anything else.
1. Design the inbox and team structure, and specify how a Fin handoff should behave so it cannot disappear. Justify each choice against the 40% figure.
Read the worked answer
Structure. One inbox per brand per channel, with teams split by brand and by function (tier 1, billing, technical). The mailbox is a routing surface, not a work queue: every inbox must have exactly one owning team and one named fallback.
Why the 40% exists. 'Unassigned' has no owner, so no one's queue length is affected by it. Nobody is measured on it, and it decays. Every handoff must therefore land in a named team's queue with an assigned state, not a shared pool.
Handoff design.
- Fin hands off with a state change (e.g. to a 'needs human' state) and an assignment to a specific team, never to unassigned.
- Pass context forward: the customer's question, what Fin already checked, which article it used, and why it stopped. Without this the agent re-asks and the customer repeats themselves - a common cause of the 'is anyone there?' reopen.
- An SLA timer starts at handoff, not at first human reply, so the wait is visible in reporting.
- A rotation fallback: if no one on the owning team picks it up within the SLA, reassign automatically to the tier-1 lead.
- Suppress further Fin replies once a human owns the conversation, so the customer is not talking to two parties.
Non-negotiable. Publish the handoff SLA internally and report handoff latency as a first-class metric. Fin's resolution rate is meaningless if the conversations it passes on rot in a queue.
2. The director wants to clean all 14,000 legacy conversations before anything else. Is that the right sequencing? What would you instrument instead, and in what order?
Read the worked answer
Cleaning 14,000 conversations first is the wrong order. Most of that history is too old to reflect current product behaviour, and the effort produces a tidy archive rather than a better helpdesk. Meanwhile every new day adds uninstrumented conversations.
Correct sequence: taxonomy → instrumentation → baseline → change.
- Define a small taxonomy. 8–12 close reasons and a controlled tag list, with the duplicates ('refund' / 'Refunds' / 'billing-issue') merged and aliased. Every reason must map to an owner who can act on it.
- Instrument going forward. Make close reasons mandatory before the conversation can be closed. This is the cheapest labelled dataset you will ever get and it is what makes Fin's performance interpretable later.
- Backfill only what you need. Sample 500 recent conversations and hand-classify them to establish a baseline. That is enough to compare month-over-month; you do not need all 14,000.
- Then change. Run the inbox restructuring, then re-baseline.
Why this matters for Fin. Without close reasons you cannot answer 'which intents does Fin fail on?' - you only see an aggregate resolution rate. The tagging work is not administrative hygiene; it is the measurement layer that makes the AI agent optimisable. Do it before the migration, because migrating 60 conflicting rules without a taxonomy just reproduces the mess in a new structure.
3. How should knowledge be organised across three brands that share most product facts but differ in tone, entitlements, and some policies?
Read the worked answer
Do not duplicate everything per brand, and do not run one undifferentiated corpus. Both fail predictably: full duplication triples maintenance and drifts into contradiction; one corpus produces answers with the wrong brand voice or the wrong entitlement.
A layered model.
- Shared core: product facts that are genuinely identical - how the API works, what the mobile app does. One source, one owner.
- Brand layers: tone, naming, packaging, and any policy that differs by brand. Separate articles, clearly owned by each brand's PM or support lead.
- Audience scoping: use audience rules so a customer on brand B does not receive brand A's entitlement language, and so internal-only or draft content is never eligible.
- Contradiction sweep: the highest-risk content is where a shared article states a policy that one brand has changed. Run a quarterly diff of shared articles against each brand's policy page.
Ownership is the real constraint. Every article needs a named owner and a review date. With nine languages, be explicit about which languages are officially covered by Fin and which fall back to English with a human handoff - an inaccurate machine translation of a health-related policy is worse than an honest 'let me connect you to a Spanish-speaking colleague'.