Agentic AI Platform
Agentic AI Platform
The Agentic AI Platform domain tests how you design, operate, and govern agents on DataRobot rather than how you prompt a single model. An agent is an LLM-driven component with instructions, tools, memory, and policies; an agentic workflow composes steps or specialized agents to complete a business task. Expect questions on orchestration patterns, tool contracts, human-in-the-loop design, and evaluation of multi-step behavior.
Core rules
- Decompose by responsibility, not by model call. A router chooses a path, a specialist performs a task, a verifier checks the result.
- Give each agent the narrowest tool set and credentials it needs. Tool descriptions are part of the contract: inputs, outputs, error modes, side effects.
- Prefer deterministic code for calculations, lookups, and policy checks; use the LLM for interpretation and language.
- Bound every loop with step, time, and cost budgets, and define a fallback path.
- Evaluate the trajectory - tool choice, arguments, order, recovery - as well as the final answer.
- Log agent steps, tool calls, and handoffs so operators can reconstruct a run.
Worked example
A support team builds a triage agent. The router reads the ticket and selects billing, technical, or escalation. The billing agent calls get_invoice and apply_credit; the technical agent calls search_kb. A policy check must run before apply_credit. The designer scores 120 historical tickets. Final-answer accuracy is 82%, but trajectory review shows 14% of credit calls used the wrong account because the tool accepted an unvalidated ID. The fix is not a better prompt: add an account-ID resolver tool, require its output as an input to apply_credit, and reject calls without it. Re-score: final accuracy 88%, invalid-credit rate 0.5%. Reasoning: the failure was a tool-contract and validation problem, visible only in trajectory evaluation.
Traps
- Defaulting to multi-agent designs when one agent with two tools is cheaper and easier to test.
- Granting broad API scopes "so the agent can recover" - this creates unsafe side effects.
- Evaluating only the final text, which hides wrong tool calls and silent retries.
- No idempotency key on write actions, so retries duplicate credits or tickets.
- Ignoring latency and token budgets; an agent that loops three times may pass quality review and fail operations.