An AI assistant trained on a stale knowledge base

The assistant's only source of truth is the knowledge base, and the knowledge base has been measured as unreliable. The audit found roughly 36 inaccurate articles out of 200 sampled, which is an 18% error rate on the sample; with 41% of articles untouched for a year, the true error rate across the whole base is unknown and probably higher, because the sample was random rather than targeted at the oldest content. Extrapolated, several hundred articles are likely to be wrong, and the wrongness is not evenly distributed: the highest-traffic articles are the ones most likely to have drifted.

An AI assistant trained on this base does not reduce the error rate, it industrialises it. It will produce confident, fluent, wrong answers at machine speed and at greater scale than 45 agents can produce, and it will do so in front of customers and with privileged instructions such as account resets. That is a materially different risk profile from a slow human reading a stale page.

The arithmetic of the claim also deserves scrutiny. Lifting first-contact resolution from 58% to 78% would exceed the target by three points. It is not impossible, but with median tenure at 9 months and heavy attrition, a meaningful share of the 58% is competence and confidence rather than lookup speed. An assistant helps an agent find an answer; it does not help them diagnose whether the question is the right one.

The prerequisites are unglamorous: a named owner and review date for every article, a measured accuracy baseline, retirement of obsolete content, article structure that separates symptom, cause and action, and telemetry linking tickets to articles used and outcomes achieved. Then a controlled pilot with a baseline and a comparison group. Sign a pilot, not a programme.

Do not start with all 3,100. Start with demand. Rank articles by usage over the last 90 days, on views, link rate from tickets, and search frequency. In most desks a few hundred articles cover the majority of contacts, so the first pass should cover the top 200 to 300 and deliver most of the benefit.

For each article in scope, apply four tests. Is it correct, checked by a quick technical review against the current environment? Does it have a named owner, a role or team rather than an individual, given 34% attrition? Has it been used recently? Is it duplicated elsewhere? Score each article and route it to refresh, merge, retire, or missing. Retirement is the fastest and cheapest win: an article that is unowned, unused for 12 months and not verified should be archived, not edited, after a spot-check to confirm nothing critical depends on it.

Build a missing list in parallel. Pull the highest-volume ticket types that have no article, because these are the ones where agents are improvising and handling time is longest.

The operating changes matter more than the clean-up. Article creation should be a step in the resolution workflow, prompted at closure when a ticket was not resolved using existing knowledge. Every article gets an owner, a review date and a version. Reviews are scheduled by risk, so privileged or security-relevant articles are reviewed quarterly and informational ones annually. Add article feedback from agents at the point of use, with a one-click this did not work that creates a task rather than a complaint. Track three measures: proportion of tickets resolved using a linked article, proportion of articles reviewed within their cycle, and article reuse rate. Without those, the base decays again within a year.

The known error database is not a separate universe. It is a filtered view of the knowledge base, holding one specific kind of content: a fault whose root cause is understood, or at least hypothesised, with a documented workaround available before a permanent fix is implemented. Its audience is the incident management process, and its purpose is to shorten diagnosis and restoration. A spreadsheet maintained by one person and invisible to the desk satisfies none of that.

Concentrating it in a spreadsheet has measurable costs. The desk rediscovers workarounds on every recurrence, which shows up as handling time and repeat contacts. The problem record receives no impact data, because incidents are not linked to it, so it cannot be prioritised. Known errors age without review, because ageing is only visible if the records are in a system that reports on them.

The redesign is straightforward in principle. Store known errors as a tagged record type in the knowledge base, keyed by affected configuration item and error signature, with fields for symptoms as a user would describe them, workaround, status, owner and review date. Surface them at the point of logging: when an agent types a symptom, suggest matching known errors before the ticket is created, and offer a one-click apply workaround that resolves the incident and links it to the problem record automatically.

Separate ownership clearly. The problem manager owns the content and lifecycle of known error records: identified, workaround documented, resolved by change, closed. The service desk owns feedback: this matched, this did not, this workaround took too long. The service desk manager owns the discipline of using it. Review known errors monthly with a report of ageing, recurrence counts and workaround effectiveness.