Council consultation summary under deadline pressure
A UK local council must summarise 2,400 responses to a 320-page draft transport plan before a committee meeting in nine days. Two analysts normally read every response; this year one is on leave. They use a general-purpose chatbot to produce a two-page summary and a themed table of objections. The draft summary is fluent, but spot checks find a statistic that does not appear in any response ('68% support'), two themes merged that respondents treated as opposites, and a minority objection mentioned once promoted to a headline. The council must publish the summary and explain how it weighed responses. The chatbot vendor's terms say outputs may be inaccurate; the council has no policy for AI use in consultations.
1. Diagnose which limitations of generative AI are visible and why they matter in a public consultation.
Read the worked answer
Three limitations are visible. First, hallucination: '68% support' is fabricated, which is dangerous in a statutory consultation record. Second, loss of semantic nuance: merging opposite themes changes what respondents meant, so the summary misrepresents the balance of opinion. Third, salience distortion: a single mention promoted to a headline overweights a minority and underweights frequency. Underlying causes include summarisation pressure (fitting 2,400 responses into two pages), lack of provenance, and the model's tendency to produce coherent narrative rather than auditable counts. A public consultation also requires fairness and explainability; a fluent summary can conceal that the council cannot show how it weighed responses. Non-determinism means re-running may produce a different summary.
2. Design a validation workflow that lets the analysts use AI without misleading the committee.
Read the worked answer
Use the model for extraction and drafting, not for final counts. Workflow: (1) ingest responses into a structured database with unique IDs; (2) use the model to tag themes and extract verbatim quotes with response IDs; (3) require every theme in the report to link to quotes and counts; (4) analysts verify a stratified sample - all headline claims, all minority themes, and a random 10% - against source text; (5) compute frequencies with deterministic code, not by asking the model; (6) run a second analyst review and record disagreements; (7) publish a methodology note stating AI-assisted triage and human verification. Add a red-team pass with deliberately contradictory responses. This keeps the efficiency gain while making the output auditable.
3. Should the council use the AI output as the basis for its final decision on the plan? Explain your reasoning.
Read the worked answer
No. The decision on a transport plan is a public-sector decision that must be taken by accountable humans, on evidence they can explain. The model can help organise and retrieve evidence, but it cannot weigh competing policy considerations, legal duties or local context, and it has already shown fabricated and distorted output. Using its summary as the decision basis would risk challenge on fairness and rationality. The committee should receive a verified summary and the underlying theme counts, with officers able to trace each claim to responses. The model's role should be disclosed and limited to triage and drafting. If the council cannot verify the summary before the deadline, it should publish a shorter verified analysis rather than an unverified AI summary.