Designing a Human-in-the-Loop AI Triage Workflow
A shared services team receives 1,400 inbound requests per month via a Gmail inbox and a web form. Requests include invoices, IT access, facilities, and HR questions, some with attachments containing personal data. The team wants monday AI to read each email, classify the request, extract due date and cost center, write a summary, assign the right person, and draft a reply. They propose an AI agent that runs on every new item and automatically sends the draft without review. The current board has 18 status labels and no required fields. Leadership is concerned about GDPR, incorrect assignments, and AI making decisions, but wants faster response. The account has 25,000 automation actions per month and is at 60%. No AI governance policy exists.
1. Design the AI workflow with appropriate human control points. Where exactly should the AI stop?
Read the worked answer
Trigger on new items from Gmail and the form. Run AI classification, extraction, and summarization as a first pass, with a confidence score and a suggested category. Deterministic automation then routes high-confidence items to the correct queue and sets an SLA timer; low-confidence or sensitive items go to a human triage queue. The AI should draft a reply into an update or draft field, not send it. A human reviews the category, extracted fields, and draft before sending, and must confirm any denial, financial commitment, or personal-data disclosure. Log the AI output, confidence, reviewer, and final action. This keeps the speed benefit in reading and routing while reserving judgment for a person. Do not let the AI agent auto-send or auto-close requests on its own.
2. Separate what should be done by AI, by automation, and by a human in this process. Give concrete examples.
Read the worked answer
AI is appropriate for unstructured language: classifying an email as invoice, IT access, facilities, or HR; extracting a due date or cost center; summarizing a long thread; and drafting a polite first response. Automation is appropriate for deterministic rules: create an item from a form, set the received date, assign a queue based on category, start an SLA timer, notify a manager when overdue, and route low-confidence items. Humans should handle policy decisions, exceptions, personal-data issues, final replies, denials, and any case where the AI confidence is low or the request is ambiguous. The boundary is not AI versus no AI; it is judgment versus repetition. If a rule can be written exactly, use automation. If it needs interpretation, use AI with review. If it creates legal or financial effect, use a human.
3. What governance and data-protection controls would you put in place before enabling this workflow?
Read the worked answer
Start with data minimization: do not send attachments or unnecessary personal data into AI prompts unless required, and redact identifiers where possible. Confirm the AI provider and data-retention terms, and document the lawful basis and retention period. Restrict access to the board and to AI-generated drafts. Run a small pilot with accuracy sampling by category, measure misclassification and extraction errors, and set a threshold before scaling. Provide a fallback path if the AI step fails, and never block the request queue. Require human approval before sending replies and before any denial or financial action. Keep an audit log of AI outputs and reviewer overrides. Add an AI usage policy covering acceptable prompts, prohibited data, and escalation. Review monthly.