AI-500 flashcards

32 cards. Read the term and the definition here. The deck at the top of this page is optional.

Supervisor orchestration pattern

A coordinator agent decomposes the request, delegates subtasks to specialist agents, and aggregates their outputs. Specialists do not call each other; control always returns to the supervisor.

Handoff orchestration pattern

Control transfers from the active agent to a peer agent judged better suited for the next step. The new agent becomes active rather than returning control to a central coordinator.

Fan-out fan-in pattern

The orchestrator dispatches independent subtasks to multiple agents concurrently, then merges all results. Best when subtasks have no ordering dependency, cutting wall-clock latency.

Agent capability registry

A catalog of each agent's name, description, and tools that the router uses to select an agent. Routing on declared metadata is more reliable than guessing from free text.

Loop termination limits

Hard caps on maximum handoffs, turns, and tool calls per request. Without them, agent-to-agent routing can cycle indefinitely, consuming tokens with no progress.

Typed agent contracts

Request and response schemas (for example JSON Schema) agreed between agents. Validation failures surface as explicit errors instead of silently malformed downstream input.

Single-responsibility agent

An agent scoped to one role, one instruction set, and one minimal tool set. Narrow scope improves routing accuracy and reduces conflicting instructions within a prompt.

Deterministic vs dynamic routing

Code-defined graphs suit workflows with known steps; model-driven routing suits genuinely dynamic paths. Deterministic flows are cheaper, testable, and easier to audit.

Azure AI Foundry Agent Service

A managed Azure service for building agents with server-side threads, runs, tool calling, and connected agents, removing the need to write custom orchestration infrastructure.

Thread and run objects

A thread stores conversation messages server-side; a run executes an agent against that thread and reports status such as in_progress, requires_action, completed, or failed.

requires_action run status

Indicates the run is paused awaiting client-side tool output. It is not a failure; the client submits results keyed by tool_call_id and resumes the same run.

Function calling round trip

The model returns tool calls with names and JSON arguments; the client executes them and returns outputs tied to each tool_call_id so the model can continue reasoning.

Connected agents

Child agents declared with a name, description, and type on a parent agent. The parent uses each description as the routing signal when delegating work.

Code Interpreter tool

A sandboxed execution tool that lets an agent run generated Python for calculation, data transformation, and file analysis, returning results and artifacts to the conversation.

File Search tool

A managed retrieval tool that indexes uploaded documents and grounds agent responses in them, handling chunking, embedding, and vector search without custom pipeline code.

OpenAPI action tool

A tool that exposes an external HTTP API to an agent via its OpenAPI specification, letting the model call operations with structured parameters.

Agent version pinning

Storing agent instructions, model version, and tool definitions in source control or infrastructure-as-code so deployments are reproducible and rollback targets are explicit.

Groundedness metric

An evaluation score measuring whether an agent's response is supported by the provided source context, catching fabricated claims that relevance and fluency scores would miss.

LLM-as-judge rubric

A fixed scoring rubric applied by a pinned judge model at temperature zero. Changing the judge model or rubric invalidates comparison of scores across evaluation runs.

Step-level evaluation

Scoring each agent's intermediate output in addition to final task success. End-to-end passes can mask a failing specialist whose errors the orchestrator compensates for.

Golden evaluation dataset

A curated set of test inputs with expected outcomes, including adversarial, ambiguous, and out-of-scope cases. Happy-path-only sets inflate measured quality.

Multi-agent token cost

Total tokens equal prompt plus completion tokens summed across every agent and turn. Fan-out delegation multiplies cost by the number of downstream agent calls.

Cumulative latency budget

End-to-end latency is orchestrator planning plus sequential agent calls plus tool round trips. Parallelizing independent branches reduces wall-clock time without changing per-call latency.

Distributed tracing for agents

Emitting a span per agent and tool call correlated by conversation or thread ID, allowing reconstruction of the full multi-agent execution path during debugging.

Handoff count metric

The number of agent-to-agent transfers per request. A rising trend signals routing confusion, ambiguous agent descriptions, or an overly broad orchestrator scope.

Managed identity for agents

An Azure Entra ID identity assigned to the agent's host so it authenticates to services without stored secrets, which models could otherwise leak from instructions.

Least privilege per agent

Each agent's tool permissions are scoped to only the resources its role requires. A shared high-privilege credential across agents removes isolation between roles.

Indirect prompt injection

Malicious instructions embedded in retrieved documents or tool outputs. Defenses treat such content as untrusted data and enforce server-side allow-lists on tool arguments.

Excessive agency risk

Danger from an agent holding broader permissions or tool scope than its task needs. Mitigate with constrained parameters and write allow-lists enforced outside the prompt.

Prompt shields and content safety

Pre- and post-filters that detect injection attempts and harmful content. Blocked events are logged for audit, and thresholds are tuned against false-positive rates.

Canary agent deployment

Releasing a new agent version to a small traffic slice and comparing task success and error rate against baseline before full rollout, with automatic rollback on regression.

Tool invocation audit log

A record of each tool call capturing caller identity, hashed arguments, and outcome. Control-plane RBAC and agent runtime permissions are separate and both must be reviewed.