A Zero-Violation KPI Across Fourteen Teams
A 14-team insurance platform has been told by its new VP of Digital that accessibility will be measured as "zero axe-core violations" by the end of Q3, 90 days away. The first scan reports 9,800 findings, dominated by duplicate component instances: 1,100 color-contrast, 2,300 missing form labels, and 800 image-alt entries, many in legacy PDFs and a shared header. There is one accessibility specialist, no dedicated budget, and no disabled users in the research panel. Three enterprise customers have sent accessibility questionnaires; the previous responses were self-asserted by product managers. One team has already shipped a "fixed" modal that traps keyboard focus. The VP wants a dashboard, weekly numbers, and a statement that the platform "is WCAG 2.2 AA compliant" by Q3.
1. Critique the zero-violation KPI and propose measures that would actually indicate progress.
Read the worked answer
The KPI measures the wrong thing and will produce the wrong behavior. Automated rules detect a minority of success criteria well; they cannot judge focus order, keyboard traps, meaningful alternative text, error recovery, or whether a screen-reader user can complete a task. A team can drive the count to zero by suppressing rules, hiding elements, or adding empty alt attributes, while the product remains unusable. The 9,800 figure also overstates the problem's shape: it is largely repeated instances of a few component defects, so reducing it is a component-library task, not 9,800 separate fixes.
Better measures, in combination:
- Component coverage: percentage of shared components that have passed a manual keyboard and screen-reader review, with a named owner.
- Task success: can disabled participants complete the top five journeys (quote, claim, policy lookup)? Measure completion and assistance needed, not satisfaction alone.
- Process measures: designs reviewed before build, accessibility acceptance criteria in the definition of done, procurement checks completed.
- Severity-weighted backlog: count open blockers and criticals, not total findings, and track age.
Frame barriers as product defects rather than user deficits; that framing changes what teams build. Report weekly, but report these measures honestly, including what is not yet tested.
2. Describe how you would triage the backlog with one specialist and no budget, and what you would fix first.
Read the worked answer
Triage on user impact, not on finding count.
First: blockers in core journeys. The keyboard-trapping modal is the highest priority because it strands keyboard and screen-reader users with no exit. Any trap, focus loss in a form, or control that cannot be operated by keyboard outranks thousands of contrast instances. Publish a short list of blocker patterns and treat them as release-stopping.
Second: systemic components. The shared header and form fields generate most duplicate findings. Fixing the form-label pattern once in the component library, with correct programmatic labels and error messaging, clears thousands of instances and prevents recurrence. Contrast fixes in design tokens similarly propagate.
Third: high-traffic, high-consequence pages. Use analytics to rank quote and claim flows above rarely visited marketing pages. Legacy PDFs are expensive; remediate only those needed for active transactions, and replace others with HTML.
The specialist's time should go to reviewing components once and writing guidance, not to hand-fixing individual pages. Train two engineers per team as reviewers so coverage scales without new headcount, and schedule a small usability study with assistive technology users to validate the triage order.
3. What should the platform tell the three enterprise customers, and what governance changes make future statements trustworthy?
Read the worked answer
Do not sign a claim that the platform is WCAG 2.2 AA compliant. A conformance claim asserts that full pages and complete processes meet the standard; with a known keyboard trap and untested journeys, that assertion cannot be supported, and a false claim creates more risk than an honest gap. Respond to customers with: the standard and version used, the scope evaluated, the method (manual plus automated, sampled pages and processes), known limitations by area, the remediation plan with dates, and a contact for follow-up. Offer to walk through specific flows the customer depends on. Distinguish self-assessment from independent audit and say which one this is.
Governance changes:
- Definition of done: accessibility acceptance criteria per component, including keyboard and screen-reader checks, verified before release.
- Design and build gates: review patterns before implementation, not after.
- Procurement: require conformance evidence from vendors and track it.
- Ownership: named accessibility leads per team, with the specialist as reviewer and coach rather than sole fixer.
- Evidence trail: keep test records so future questionnaires are answered from data, not opinion.
- Feedback channel: a route for disabled users to report barriers, with tracked responses.
Set a realistic horizon. Component-level coverage in two quarters and a full audit later is credible; "AA by Q3" is not.