I85PREP
⌘ K
AI Business Professional study materials
AI Business Professional study resources

The Prompt That Wrote a Policy

AI Business Professional · Put your knowledge to work in context.

All case studies

The Prompt That Wrote a Policy

An HR operations lead uses a public generative AI chatbot to draft a new remote-work policy. She pastes in an internal memo that includes employee names, salary bands, and two pending disciplinary cases, plus a paragraph of her own commentary. She iterates over ~40 prompts, then pastes the final draft into the company wiki under her name. A week later an employee asks a question that reveals the policy contradicts an existing leave policy. Separately, a colleague notices the chatbot vendor's terms allow using inputs to improve its models unless the user opts out, which no one did. The company has no AI-use policy, and the HR lead believes she did nothing wrong because 'the output was just a draft I edited.'

1. Enumerate the distinct risks this single episode created, separating them into data/privacy, accuracy, and governance categories.

Read the worked answer

Data / privacy risks:

  • Personal and sensitive data disclosure. Employee names, salary bands, and pending disciplinary cases were pasted into a third-party tool. Salary and disciplinary data are among the most sensitive HR data; disclosure to an external vendor is a potential privacy incident requiring assessment and possibly notification, depending on jurisdiction and contract.
  • Model-training exposure. The vendor's terms allow input reuse unless opted out, and no one opted out. The data may now be in a training corpus - effectively irreversible. This is the most serious consequence because it cannot be undone by deleting a chat.

Accuracy risks:

  • Contradiction with existing policy. The draft conflicts with the leave policy. A generative model has no knowledge of the company's actual policy corpus, so it invented plausible-sounding terms. The human edit did not catch a cross-document conflict because no one checked against the source of truth.
  • Fabricated specificity. Policies drafted this way often contain confident but invented details (notice periods, eligibility) that read as authoritative.

Governance risks:

  • No AI-use policy. There was no rule to violate, which is itself the failure - the organization left the judgment to an individual.
  • Unreviewed publication under a name. The draft went live without legal/compliance review, creating apparent organizational commitment to terms no one approved.
  • False sense of safety. 'It was just a draft' misunderstands that the risk is in the input and the publication, not only the output.

2. The HR lead says she 'did nothing wrong because it was just a draft.' Respond to that reasoning and specify the minimum controls the company should put in place.

Read the worked answer

Why the reasoning fails: 'It was a draft' addresses only one risk (output quality) and ignores the two larger ones. The input contained sensitive personal data sent to an external service - that exposure happened at paste time, before any draft existed. The publication put unapproved terms into force - that happened at wiki-post time, and the edit did not prevent a cross-policy contradiction. A draft can still cause a privacy incident and an operational conflict; 'draft' is a stage, not an absolution.

Minimum controls:

  1. Data-handling rule for AI tools. Prohibit pasting personal, salary, disciplinary, health, or client-confidential data into unapproved tools. Provide an approved tool with a no-training / data-retention agreement, and a sanctioned way to do HR drafting.
  2. Opt-out and vendor review. Default to vendors whose terms exclude training on inputs; where not possible, configure opt-outs and record it. Procurement should review AI terms, not just price.
  3. Human review against source of truth. Any AI-assisted policy must be checked against the existing policy corpus and approved by legal/compliance before publication. 'Edited by a human' is not the same as 'verified.'
  4. Label and own AI-assisted content. Mark drafts as AI-assisted internally and route them through the normal approval path.
  5. Incident path. Treat the data exposure as a reportable event to assess, not a lesson to shrug off - determine whether notification or remediation is required.
  6. Training. The lead's belief is common; the fix is a short, concrete rule with examples, not a policy PDF nobody reads.