Undocumented Rules and a Retiring Expert: Choosing Elicitation Techniques
ClaimsCo is replacing a 14-year-old claims adjudication system. The 26 claims assessors work to undocumented rules embedded in the legacy code and in the heads of two senior specialists, one of whom retires in four months. Three business regions - Manila, Leeds and Calgary - must contribute, and assessors cannot leave the floor during the 09:00 to 17:00 processing window. The programme manager ran a two-day requirements workshop with 22 attendees and produced 500 items, most of which read like screen designs. Two weeks later the vendor's architect described the list as unbuildable and contradictory. A further complication: the regional heads disagree on whether assessors have discretion to deviate from the rule, and that disagreement changes the rules materially.
1. Select and justify the elicitation techniques you would use for each of the distinct information needs in this situation.
Read the worked answer
Match technique to the type of information required; no single technique covers all of these needs.
- Undocumented legacy adjudication rules - document analysis first (code, job logs, help text, historic exception reports), then observation and job shadowing at the desk. Rules embedded in behaviour are exposed by watching how exceptions are handled, not by asking 'what are the rules?'
- Regional disagreement over assessor discretion - facilitated workshops per region, followed by a cross-region decision session. This is a policy conflict rather than a gap in facts, so it needs a forum with decision authority. Interviews first would produce three incompatible private versions.
- High-volume factual data (exception frequency, case mix, volumes by claim type) - survey or direct data extraction from the legacy database. Do not spend workshop time on facts you can measure.
- Tacit knowledge held by the two senior specialists - structured interviews combined with observation and think-aloud walkthroughs of real historical cases, plus a focus group of the two together so they correct each other. Add analysis of their informal notes.
- Confirmation of elicited rules - review and inspection with the assessor group, using concrete cases ('claim A with these attributes - what should happen?') rather than abstract statements.
Constraints shape the mix. Assessors cannot leave the floor during processing hours, so desk-side shadowing beats off-site workshops. Three time zones mean short, scheduled sessions with pre-circulated rule drafts. Reserve workshops for conflict resolution and consensus - that is where they add value other techniques cannot.
2. Diagnose why the 22-person workshop produced 500 unusable items, and describe how you would design the replacement session.
Read the worked answer
Diagnose the failure before redesigning.
Why 500 items emerged
- 22 attendees with no decision rights: at that size a workshop is a presentation, not elicitation.
- No stated purpose or scope boundary, so participants produced screen designs. Solutioneering fills the vacuum left by undefined business rules.
- No pre-work: assessors arrived without data on their own exception volumes.
- No scribe discipline or parking lot, so every idea became a 'requirement'.
- Output was never validated, so contradictions surfaced only at the vendor's review.
Replacement design
- Pre-engage: interview the three regional heads individually to surface the discretion conflict, and obtain a written decision or escalate it before the workshop - otherwise it will consume the session.
- Scope: one workshop per process area (for example, 'first notification and validation'), 8 to 10 participants who can commit their area's rules.
- Roles: named facilitator, scribe, a business owner with authority to accept, and you holding the model.
- Pre-work: a one-page draft rule set and three anonymised real cases circulated five days ahead.
- Agenda: timeboxed, with decisions listed up front and a visible parking lot for solution ideas.
- Output: rule statements in a structured template with owner, source and status (proposed, agreed, disputed) - not prose bullets.
- Follow-up within 48 hours: circulate for confirmation, log disputes, assign owners.
The 500-item list should not be discarded wholesale. Triage it into genuine rules, open questions and design ideas, then use it as the draft for validation. That converts sunk cost into an asset and avoids re-asking SMEs what they have already told you.
3. How would you capture the retiring specialist's knowledge and confirm that elicited results are correct and accepted?
Read the worked answer
Treat the retiree as a time-boxed single point of failure: knowledge capture is a scheduled deliverable with a hard date, not background activity.
Capture plan
- Book 16 weeks of his calendar now, with a named backup BA in every session.
- Work from real cases: select 30 to 40 adjudications spanning the exception types and have him narrate the decision path for each. This elicits thresholds and rules that abstract questioning will not surface.
- Record and transcribe with consent; produce a decision table or rule model per case and review it back to him within a week while context is fresh.
- Pair him with a successor SME throughout - the objective is transferred competence, not just a document.
- Track coverage as exception types modelled against a list he validates as complete. Stop when he confirms completeness and the successor can apply the rules unaided.
- Convert anything unresolved into explicit open issues with an owner and a decision date. Silence is not agreement.
Confirming results
- Verification: is the rule stated correctly and unambiguously? Check it against historical case data and replay it against past outcomes.
- Validation: does the rule support the business objective, and is it accepted by the accountable business owner rather than only by the expert?
- Run a formal review with the assessor group and a walkthrough against test cases, recording sign-off per rule set so acceptance is traceable.
Escalate immediately if the 16 weeks is not protected - that is a dated project risk, not a resourcing preference.