Scaling a generative AI pilot in a utilities contact centre
You are operations manager at a utilities company with 400 employees. A pilot has used generative AI to draft responses to customer emails for six weeks with 12 agents. Average handling time is down 18%, but a sample audit found that 7% of drafts contained incorrect policy details, and two drafts included a previous customer's complaint from a different account. Agents say they are "double-checking everything anyway". The vendor contract states that customer data is used to improve the model unless the company opts out, and legal confirms the opt-out has not been signed. The staff council wants consultation before any expansion. The board wants to scale to 120 agents in eight weeks. There is no AI use policy.
1. What risks must be controlled before any expansion, and how would you sequence the controls?
Read the worked answer
The pilot has produced a productivity signal but not a safety case. The 7% policy error rate is material in utilities, where a wrong statement about tariffs, arrears or priority services can cause financial or safeguarding harm. The two cross-account disclosures are a data protection incident, not a quality defect. The unsigned opt-out means customer data may already be flowing into model improvement without a lawful basis. These are not risks to note in a slide; they are stop conditions for expansion.
I would sequence controls in this order. First, freeze any further data sharing with the vendor and get legal to confirm the contractual position and whether a retrospective opt-out or data deletion is required. Second, report the cross-account incidents through the company's data breach process and notify affected customers if the law requires it. Third, suspend the pilot's use of live customer data until a sandbox or redaction layer is in place. Fourth, introduce a mandatory human-review rule: no AI draft can be sent without an agent checking policy details against a controlled source, and high-risk categories such as arrears, vulnerability and complaints must be drafted manually. Fifth, create a temporary AI use policy covering permitted tools, data classification, logging, review and escalation.
I would not scale to 120 agents in eight weeks. I would propose a gate: two weeks of sandbox testing, then a controlled extension to 20 agents for four weeks with 100% audit of high-risk drafts and 20% sampling of others. Expansion beyond that would depend on error rate below 2%, zero cross-account disclosures, signed contractual terms, and staff council agreement. The board's timeline is a business aspiration; the sequence above turns it into a defensible decision.
2. How would you involve the staff council and agents without turning consultation into a blocker?
Read the worked answer
Consultation is not a vote on whether AI is used; it is a structured conversation about how it changes work, risk and accountability. I would start by giving the staff council the same evidence I have: the 18% handling-time reduction, the 7% error rate, the two data incidents, and the unsigned opt-out. I would be explicit that I am not asking them to approve the pilot, but I am asking for their input on safeguards and implementation. That distinction prevents the conversation from becoming a binary yes/no.
I would then run two parallel sessions. With the staff council, I would focus on consultation requirements, monitoring, job design and how agents can raise concerns without blame. With the 12 agents, I would run a short workshop to map where the tool helps and where it creates double-checking. The comment that agents are "double-checking everything anyway" is important: if the tool saves 18% handling time but adds a verification burden, the net benefit may be smaller than reported, and agents may be compensating for known errors.
To avoid blockage, I would agree a time-bound consultation: two weeks, with a clear decision date and a named owner. I would publish a summary of what changed as a result of their input. I would also create an escalation route so agents can report a bad AI draft without being blamed for the tool's error. If the staff council raises a legitimate concern that cannot be resolved in two weeks, I would extend only that issue, not the whole programme. This keeps momentum while showing that consultation has real influence.
3. What metrics would you use to judge whether the AI is safe and worthwhile?
Read the worked answer
I would use a balanced set, because handling time alone would reward speed at the expense of accuracy. The safety metrics would be: policy error rate from audit, with a target below 2% before expansion; cross-account or personal-data disclosure incidents, with a target of zero; and the percentage of drafts sent without human review, which should be zero for high-risk categories. I would also track complaints coded as "incorrect advice" and any regulatory or legal escalations.
The quality metrics would be: first-contact resolution, repeat contact rate within seven days, and quality assurance score for AI-assisted emails compared with a matched sample of manual emails. The "double-checking" comment needs a direct measure, so I would time the verification step separately from drafting. If verification adds more time than drafting saves, the 18% handling-time reduction is partly an artefact.
The workforce metrics would be: agent confidence, measured by a short pulse survey; reported near-misses; and voluntary turnover in the pilot group. The commercial metrics would be: cost per contact, backlog reduction, and customer satisfaction. I would set a review at four weeks with pre-agreed thresholds. If safety thresholds are met but commercial benefit is weak, I would keep the tool as an agent-assist rather than scale it. If safety thresholds fail, no commercial gain justifies expansion.