AI-500 study plan

Diagram a multi-agent orchestration pattern (supervisor + specialists) using Azure AI Foundry Agent Service concepts, labeling each agent's role, tools, and handoff points

Provision a sandbox resource group and register at least two agents with distinct instructions and tool access

Implement a basic routing/transfer flow between agents and log each handoff with correlation IDs

Write a short design note comparing sequential, concurrent, and handoff orchestration for one business scenario

Add tool/function calling to at least one agent and validate input/output schemas

Integrate a retrieval source (vector index or search) so an agent grounds answers in supplied documents

Implement conversation state/thread management so context persists across turns

Handle and log at least three failure modes (tool timeout, malformed tool output, empty retrieval)

Refactor the demo into a supervisor pattern where a coordinator agent delegates to two specialist agents

Define explicit contracts (input/output schema and escalation rules) for every agent-to-agent boundary

Implement retry, timeout, and fallback behavior for inter-agent calls

Document the orchestration topology and justify each design choice against alternatives

Define evaluation metrics (task success, groundedness, latency, cost per run) for the multi-agent solution

Build an evaluation dataset of at least 20 labeled cases covering normal, edge, and adversarial inputs

Run an automated evaluation pass and record baseline scores per metric

Instrument tracing so every agent step, tool call, and token count is captured

Identify the three lowest-scoring cases and diagnose root cause in the trace (prompt, tool, routing, or retrieval)

Tune prompts, routing rules, or retrieval settings and re-run the full evaluation to measure improvement

Add cost and latency budgets per agent and flag runs that exceed them

Configure alerting or dashboards for failure rate, latency, and token consumption

Configure identity and least-privilege access so each agent uses a managed identity scoped to only its required resources

Apply content safety and input/output filtering to every agent boundary and test with adversarial prompts

Implement secret-free configuration and verify no credentials appear in code or logs

Map each agent's data flows and document where sensitive data is stored, transmitted, and logged

Implement governance controls: logging retention, audit trail, and responsible-AI review notes for the solution

Package the solution for deployment with infrastructure-as-code and environment-specific configuration

Deploy to a staging environment and run the full evaluation suite against the deployed version

Verify monitoring and alerts fire correctly in the deployed environment

Conduct a full architecture review of the solution, documenting tradeoffs across all four domains

Complete two timed practice runs that force decisions on orchestration, evaluation, and security tradeoffs

Close knowledge gaps by re-testing the weakest domain from prior weeks

Write a final design document and retrospective summarizing what you would change