Operational Control of an Internal Generative-AI Assistant
Helios Bank has deployed an internal generative-AI assistant called Atlas to 1,500 employees. Atlas is connected to SharePoint, the email archive and the IT ticketing system, and is supplied by a SaaS vendor that pushes model updates monthly without prior notice. In the first quarter, three incidents occurred: an employee pasted customer account numbers into a prompt, a hallucinated answer told staff to waive a fee that policy did not allow, and a team discovered that prompts were retained for 30 days by the vendor but not logged internally. There is no acceptable-use policy beyond an email reminder, no approval process for new data connectors, and no human review step for Atlas answers used in customer communications. The vendor's contract has no AI-specific clauses. The COO wants to expand Atlas to the contact center in 30 days. The CISO wants to switch it off. The AIMS implementation team must decide.
1. Identify the controls and operational arrangements needed before any expansion, mapping them to the AIMS implementation and Annex A control areas.
Read the worked answer
Map the gaps to control areas: AI system inventory and classification; acceptable use and data handling; access control and data loss prevention; logging and monitoring; human oversight; supplier management; change management for model updates; incident management; competence and awareness. Concrete arrangements: a policy that prohibits customer identifiers in prompts and defines permitted data; prompt filtering and data loss prevention; retention and deletion rules agreed with the vendor; immutable logging of prompts and outputs with access controls; a human review requirement for any customer-facing output; a connector approval board with risk assessment; a vendor schedule of changes and notification; a kill switch and fallback; and incident tickets with root cause. Do not expand to the contact center until these are operating. The three incidents show that current controls are not effective: the data leak is a prevention failure, the hallucination is an output-verification failure, and the missing logs are a monitoring failure. Each needs corrective action, not just a reminder. The Statement of Applicability should show which Annex A controls are selected and why. The COO's expansion should be treated as a change requiring risk assessment and management approval.
2. How should Helios handle roles, competence and documented information for Atlas? Be specific about what evidence an auditor would expect.
Read the worked answer
Roles should include an accountable AI system owner for Atlas, a product and operations owner, a risk owner, a data steward, the DPO, security and internal audit independent of operation. Define who can approve new connectors, who reviews outputs, who handles incidents and who can shut it down. Competence: all 1,500 users need awareness of permitted use, data classification and hallucination risk; approvers need training on verifying outputs; engineers need training on logging, testing and change control. Keep a competence matrix and records of training and assessment. Documented information should include the AI policy, acceptable-use procedure, risk and impact assessments, Statement of Applicability, supplier assessment, change records for monthly model updates, incident records, logs, training records and management review minutes. An auditor will not just read the policy; they will sample prompts and logs, ask users what they do with a suspicious answer, check whether connector approvals exist and trace an incident to corrective action. Evidence should be retrievable by system and date. If training records are only an email reminder, competence is not demonstrated.
3. What is the appropriate decision on the contact-center expansion and why? Consider risk treatment, monitoring and management responsibility.
Read the worked answer
Expansion should be refused or limited until the baseline is effective. Contact-center use involves customer data and external communications, so the impact of a hallucination or leak is higher. The decision is a risk treatment decision: avoid by pausing, reduce through a pilot with human-in-the-loop and no sensitive data plus enhanced logging, transfer through contract changes, or accept with documented top-management approval. The COO's 30-day target is not evidence of readiness. A defensible path is a limited pilot with a small group, mandatory human review, no customer identifiers, enhanced logging, daily review of failures and a stop criterion. The CISO's 'switch off' is also a legitimate avoid option, but it may be disproportionate if controls can be implemented. What matters is that top management makes an informed decision based on the risk assessment, not that one faction wins. Record the decision, owner, review date and residual risk. Before certification, show that the chosen controls operate and that incidents led to improvement.