AI Coding Assistant Rollout in a Regulated Bank
A bank with 3,000 developers pilots an AI coding assistant. Within two weeks, developers report it suggests plausible but subtly wrong API calls, and several pull requests include generated code that passed review because reviewers trusted the tool. The bank's change management policy requires peer review but says nothing about AI-generated code. Separately, a developer pasted a customer data schema into the assistant's chat to get help with a query; the vendor's terms allow limited use of inputs for model improvement unless the enterprise tier is purchased. The bank is under a regulatory consent order for data handling. The CISO wants to ban the tool; the CTO argues a ban will push developers to unsanctioned tools. The pilot has 90 days remaining and no evaluation metrics were defined.
1. Evaluate the ban-versus-continue decision. What is the defensible path?
Read the worked answer
A blanket ban is likely to fail on its own terms: developers will use personal accounts or unapproved tools, which is worse because those are invisible to security. Continuing unchanged is also indefensible given the consent order and the data-pasting incident.
The defensible path is conditional continuation with controls: purchase the enterprise tier so inputs are not used for model improvement and data handling terms are acceptable; restrict the tool to code contexts and block pasting of customer data through policy and technical controls; require disclosure when AI assistance was used in a change; and define evaluation metrics now - defect rate, review findings, incident count - rather than at day 90.
The judgment point is that the CTO is right about shadow use and the CISO is right about exposure. The answer is a governed middle path with measurable gates, not a binary choice.
2. What must change in change management and code review to address subtly wrong generated code?
Read the worked answer
Peer review assumes the reviewer independently verifies logic; when reviewers trust a tool, that assumption breaks. Changes should require the author to identify AI-assisted portions and to state how the code was verified - tests, documentation checks, or API contract validation.
Add automated verification that does not depend on reviewer vigilance: expanded unit and integration tests, static analysis, and dependency and API validation in CI. Reviewers should be accountable for verification evidence, not just approval.
Track defect and rollback rates for AI-assisted versus human-authored changes to detect drift. If the data shows higher defect rates in a category, tighten review for that category.
The judgment point is that the control must not rely on the same trust that the tool eroded. Verification should be mechanical wherever possible.
3. How should the bank handle the data-pasting incident under the consent order?
Read the worked answer
Treat it as a potential data handling event, not a training issue. Determine what was pasted, whether it contained customer identifiers, and what the vendor's terms permit; document the timeline. Engage legal and compliance to assess whether notification or regulatory disclosure is required under the consent order.
Remediate technically: block paste of sensitive patterns, use enterprise terms that prohibit training on inputs, and log tool usage. Communicate to developers why the boundary exists, since the developer likely saw the tool as a productivity aid, not a data transfer.
The judgment point is that a consent order raises the bar from "best practice" to "demonstrable control." The bank must be able to show the regulator that it detected, assessed, and corrected the exposure.