NVIDIA NCP-AAI flashcards

32 cards. Read the term and the definition here. The deck at the top of this page is optional.

ReAct agent pattern

An agent design that interleaves explicit reasoning steps with tool calls, using each observation to update the next thought. It improves transparency and error recovery compared with single-shot planning.

Supervisor orchestration

A topology where a router agent decomposes the request, delegates subtasks to specialist agents, and aggregates their outputs. It suits heterogeneous subtasks but adds a coordination hop and token cost.

Plan-and-execute agent

An agent that produces a complete multi-step plan before acting, then executes steps and replans only on failure. It reduces per-step reasoning cost but is brittle when early assumptions break.

Handoff summary payload

A compact structured message passed between agents containing goal, constraints, findings, open questions, and artifact references. It prevents context growth that occurs when full transcripts are relayed.

Tool description as selection criteria

Writing a tool's description to state when it should and should not be chosen, rather than documenting its internals. This directly reduces wrong-tool selection by the planner.

Fan-out fan-in merge

Parallel dispatch of independent subtasks followed by a deterministic merge step that deduplicates and resolves conflicts. It is safe only when subtasks share no mutable state.

Agent memory tiers

The layered store set: in-context scratchpad for ephemeral state, vector store for semantic recall, structured database for durable facts, and episodic logs for replay and audit.

Delegation loop cap

A hard limit on agent iterations or handoffs per task. It bounds runaway token spend and latency caused by circular delegation between agents.

Tokens per completed task

A cost metric summing input and output tokens across every hop of a task. It exposes multi-agent cost multiplication that per-request metrics hide.

Prompt prefix caching

Reuse of computed key-value state for an unchanged prompt prefix. Placing stable system prompts and tool schemas first and volatile content last maximizes cache hits and cuts prefill cost.

Prefill versus decode latency

Prefill processes the whole prompt in parallel and scales with input length; decode generates tokens one at a time and scales with output length. They require different optimization tactics.

Model cascade routing

Sending easy queries to a smaller, cheaper model and escalating only hard cases to a larger one. It lowers average cost and latency while preserving quality on difficult inputs.

Backoff with jitter

A retry strategy where wait time grows exponentially and a random offset is added. Jitter prevents synchronized retry storms from overwhelming a rate-limited endpoint.

Stateless agent workers

Agent service instances that keep no session state locally, storing it externally instead. This enables horizontal autoscaling without sticky sessions or affinity routing.

NIM microservice scaling

NVIDIA inference microservices packaged as containers with OpenAI-compatible APIs, scaled on Kubernetes. GPU memory for concurrent KV cache, not CPU, is the binding capacity constraint.

Per-hop tracing

Instrumenting each agent step and tool call as a separate span with token and latency attributes. Without it, cost and latency regressions in multi-agent systems remain invisible.

Graceful degradation policy

A predefined fallback ladder such as shorter context, smaller model, or cached answer, applied when quotas or latency budgets are exceeded instead of returning a hard failure.

Least-privilege agent tooling

Granting each agent only the scopes required for its task, with short-lived scoped credentials. It limits blast radius when an agent is manipulated or behaves incorrectly.

Indirect prompt injection

Malicious instructions embedded in retrieved documents or tool outputs that the agent treats as commands. Defense requires separating instructions from data and never widening permissions from tool output.

Human-in-the-loop approval

A control requiring explicit human confirmation before an agent executes an irreversible action such as payment, deletion, or external message. It trades latency for accountability.

Human-on-the-loop monitoring

A model where the agent acts autonomously while a human observes and can intervene. It suits reversible actions with limited blast radius and requires reliable alerting.

AI system inventory

A governance register listing each deployed AI system, its purpose, risk classification, owner, data sources, and evaluation status. It is the anchor artifact for audits and incident response.

Prompt and model registry

Version-controlled records of prompt templates, tool schemas, and model endpoints tied to evaluation results. It enables reproducible rollback and traceable change review.

Disclosure obligation

The requirement to tell users they are interacting with an AI system and to provide an explanation path for decisions that adversely affect them.

Red-team regression suite

A fixed set of adversarial prompts covering jailbreaks, injection, and data exfiltration, run on every prompt or model change with deployment gated on pass rates.

Data minimization in prompts

Including only the personal data a task requires, redacting before logging, and honoring deletion across vector stores and caches. It reduces breach impact and retention risk.

NIM inference microservice

A containerized NVIDIA model server exposing an OpenAI-compatible API. It lets agent code target a standard interface while models scale independently on GPU infrastructure.

Hybrid retrieval with reranking

Combining dense vector search with keyword search, then reranking the candidate set with a cross-encoder. Reranking usually improves precision more than changing embedding models.

RAG versus fine-tuning

Use retrieval for changing facts and citations; use fine-tuning for stable style, format, or tool-calling behavior. Fine-tuning does not reliably inject new factual knowledge.

Trajectory evaluation

Scoring the sequence of steps an agent took, not just the final answer, to detect invalid tool paths, redundant calls, and unsafe actions that still produced acceptable output.

Grammar-guided decoding

Constraining token generation to satisfy a JSON schema or formal grammar, guaranteeing parseable structured output. Validation and repair logic should still run afterward.

Atomic version tuple

Deploying prompt, tool schema, and model endpoint versions together as one unit and logging the tuple per trace, so a regression can be reverted in a single step.