Google IT Support study materials
Google IT Support study resources

The Chatbot Wrote the Runbook

Google IT Support · Put your knowledge to work in context.

All case studies

The Chatbot Wrote the Runbook

You administer a 500-user Microsoft 365 and Active Directory environment. Your team wants to publish 40 knowledge-base articles by Friday and proposes having a public AI chatbot draft them from recent ticket notes and generate a PowerShell script that disables AD accounts inactive for 30 days and moves them to a Disabled OU. A junior technician has already pasted a ticket into the chatbot containing a user's national ID number and medical leave dates. The draft script includes a line that removes every user in an OU and calls a cmdlet that does not exist. Your manager asks whether AI can just handle it.

1. What do you do about the pasted ticket, and what usage rule do you set for the team?

Read the worked answer

Treat the paste as a data incident, not a training mistake.

  1. Contain and report: follow your data-handling policy. Notify security or privacy, record what was pasted, when, by whom, and which tool, and determine whether the tool retains or trains on inputs. Assume it may.
  2. Do not blame the junior publicly. They used the tool the team was implicitly encouraged to use. Fix the process, not just the person.
  3. Change the tool, then the habit. Move drafting to an enterprise AI tenant with contractual no-training and data-protection terms if one exists. If none exists, restrict use to non-sensitive text only.

Standing rule for the team:

  • No names, IDs, addresses, medical details, credentials, or customer data in prompts. Replace with placeholders like USER_A and TICKET_1234.
  • Paste the shape of the problem, not the record: 'a ticket where the user reports X, with fields A, B, C' is enough for a draft.
  • Any AI output that touches a real account or mailbox needs human review before it is used.

Reasoning: the value of AI here is drafting speed. None of that value requires sending personal data to an external service.

2. How do you validate the generated AD cleanup script before it runs anywhere near production?

Read the worked answer

Treat the script as an untrusted draft from a confident stranger.

  1. Read it line by line. The Remove-ADUser with a broad filter and the nonexistent cmdlet are exactly the failure modes that matter: one destroys accounts, the other fails silently or aborts mid-run.
  2. Fix the logic before testing: filter on last logon using the replicated attribute (LastLogonTimestamp), scope with -SearchBase to the specific OU, exclude service accounts, disabled accounts, and a named exception group, and replace removal with disable plus move.
  3. Test with -WhatIf first and read the full output list. Then test in a lab or a test OU containing deliberately planted accounts: one active, one 45 days idle, one service account, one already disabled.
  4. Add guardrails: a hard count check (abort if more than N accounts match), transcript logging, and a rollback path (re-enable and move back from a recorded CSV).
  5. Change control and peer review: a second admin reviews, the run is scheduled, and the output list is approved before execution. Run under a least-privilege account, never a domain admin.

Rule: AI can draft the script; it cannot approve it. The reviewer owns the outcome.

3. Design the workflow for the 40 articles so Friday is realistic and safe. What do you tell your manager?

Read the worked answer

Tell your manager: AI can cut drafting time substantially, but it cannot own accuracy, and publishing unreviewed AI text into a knowledge base creates a worse problem than an empty knowledge base.

Workflow:

  1. Select and redact: pick the 40 tickets, strip identifiers, and keep the technical facts.
  2. Draft with AI: generate an outline and first draft per article from redacted notes. Batch this; it is the fast part.
  3. SME review: the technician who resolved the issue verifies every command, path, and version detail against official documentation or a tested environment. Unverified commands do not ship.
  4. Publish with a label: mark AI-assisted drafts, record the reviewer and date, and keep the source ticket link.
  5. Triage by risk: articles containing commands or destructive steps get full review; purely explanatory articles can have a lighter review.

Realism check for Friday: if 40 full reviews are not feasible, publish the highest-value 15 to 20 properly and queue the rest. A smaller accurate set is better than 40 articles that send people to the wrong OU.

What to avoid: auto-publishing anything, and using a benchmark of 'articles produced' as the success metric instead of 'articles that resolved a ticket without escalation.'