NVIDIA NCP-AAI common mistakes
NVIDIA NCP-AAI · Understand the traps and how to avoid them.
Designing a single monolithic agent that performs planning, retrieval, tool calls, and final response generation in one prompt, then failing scenario questions that ask how responsibilities should be partitioned across planner, executor, and critic agents.
Candidates carry over habits from single-prompt LLM application design and treat multi-agent architecture as an optional refinement rather than a structural requirement. They also underestimate how a single agent's context degrades as tool outputs accumulate.
What to do instead
Practice decomposing a task into explicit roles: an orchestrator that decomposes goals and routes, worker agents with narrow tool scopes, and a verifier that checks outputs before return. For each scenario, justify the split by context isolation, independent failure handling, and testability, not by novelty.
Choosing a fixed sequential pipeline for every workflow, even when the task requires conditional branching, retries, or dynamic tool selection, and then being unable to explain how the design recovers when a mid-step tool call fails.
Sequential chains are the easiest pattern to describe, so candidates default to them and do not rehearse alternative topologies such as supervisor, hierarchical, or handoff-based routing.
What to do instead
Map each scenario to a topology and state the selection criteria: determinism and auditability favor explicit graphs; open-ended subtasks favor supervisor delegation; latency-sensitive paths favor parallel fan-out with aggregation. Always describe the failure path, including retry limits and fallback agents.
Treating agent-to-agent message passing as free-form natural language chat without defining a structured contract for payloads, tool schemas, and termination signals, then missing questions about loops, deadlocks, or ambiguous handoffs.
Candidates focus on the intelligence of individual agents and neglect the protocol layer, assuming the model will resolve ambiguity at runtime.
What to do instead
Specify typed message schemas, explicit task-completion and abort signals, maximum hop counts, and idempotency keys for tool calls. When reviewing a design, ask what stops two agents from ping-ponging indefinitely and how a partial result is escalated.
Answering scalability questions by proposing to scale the model or add more agents, without addressing token growth, context window limits, latency budgets, or cost per task.
Candidates conflate model capability with system throughput and have not internalized that multi-agent designs multiply token consumption through inter-agent chatter and repeated context.
What to do instead
Quantify the design: estimate tokens per step, number of steps, and calls per task; then propose concrete levers such as context summarization, retrieval instead of full-context stuffing, caching of stable prefixes, parallel execution of independent subtasks, and tiered models where cheap models handle routing.
Ignoring state management and durability, so scenarios involving long-running tasks, human approval waits, or process restarts are answered as if the agent holds everything in memory.
Prototyping environments rarely crash mid-task, so candidates never confront checkpointing, resumability, or session persistence.
What to do instead
Describe an explicit state store for task progress, tool results, and pending approvals, with checkpointing after each significant step. Explain how a restarted worker resumes from the last checkpoint and how duplicate side effects are prevented via idempotency.
Proposing horizontal scaling of agents without addressing shared resource contention such as rate limits on tool APIs, vector database throughput, or concurrent writes to a shared memory store.
Candidates think of agent replicas as stateless web servers and overlook that downstream tools and memory services become the bottleneck.
What to do instead
Identify the constrained resource in each scenario and propose queueing with backpressure, per-tenant rate budgets, read replicas or caching for retrieval, and partitioned memory namespaces. State the metric used to detect saturation, such as queue depth or tool error rate.
Treating guardrails as a single input filter and failing questions about tool-level authorization, output validation, and privilege boundaries between agents.
Candidates equate safety with prompt filtering and do not consider that an agent with tool access can take real actions that bypass text-level checks.
What to do instead
Layer defenses: input classification, least-privilege tool scopes per agent, allowlists for actions, human confirmation for irreversible operations, and output validation before any downstream action. Describe where each layer sits in the request path.
Designing oversight as a blanket human-in-the-loop requirement for every step, then failing scenarios that test risk-tiered approval or that penalize excessive latency and alert fatigue.
Candidates equate more human review with better governance and do not distinguish reversible from irreversible actions.
What to do instead
Tier actions by reversibility and blast radius: auto-approve reads and drafts, require confirmation for writes and external communications, and require dual approval for high-impact operations. Justify each tier and describe the audit record captured at each level.
Describing evaluation only in terms of final answer quality, omitting trajectory-level checks such as tool selection accuracy, unnecessary steps, policy violations, and hallucinated tool arguments.
Candidates transfer evaluation habits from QA-style LLM benchmarking, where only the output text is scored, and are unfamiliar with trajectory evaluation for agents.
What to do instead
Define metrics at both levels: task success and groundedness for outcomes; step count, tool-call validity, and policy compliance for trajectories. Include adversarial and red-team cases, regression suites for prompts and tools, and a process for promoting production failures into test cases.
Proposing a deployment plan that jumps from prototype to production without observability, versioning of prompts and tool schemas, or a rollback path, and then failing questions about diagnosing a regression after a model or prompt change.
Candidates treat deployment as a one-time launch event rather than a continuously versioned system, and they underestimate how prompt edits silently change agent behavior.
What to do instead
Specify tracing of every step with inputs, outputs, latency, and cost; version prompts, tool schemas, and model endpoints together as a release unit; use canary rollout with shadow traffic; and define rollback triggers tied to success-rate and cost metrics.