Generative AI application development and orchestration
Domain 6 - Generative AI application development and orchestration
Application-development questions test whether you can turn a prompt into a reliable service: orchestration, tools, guardrails and failure handling.
Building blocks
- Prompt templates with typed variables, versioned separately from code.
- Model inference via the watsonx.ai Python SDK (ModelInference, then generate_text, chat, or generate_text_stream) or the REST API.
- Tools / function calling to reach databases, search and enterprise systems.
- Agents that plan, call tools, observe results and iterate.
- Guardrails on input and output: PII, toxicity, prompt injection, schema validation.
- Orchestration layers: watsonx.ai flows for visual pipelines, watsonx Orchestrate for enterprise skills, watsonx Assistant for conversational front ends, plus LangChain integrations.
Engineering rules
Stream tokens for perceived latency. Set timeouts, retries with backoff, and a hard cap on agent iterations. Budget tokens per request. Cache deterministic calls. Log prompt version, model version, token counts and retrieved document ids for every call.
Worked example
An agent must answer billing questions. Flow: classify intent → retrieve from billing docs → draft → validate that every claim is grounded in a retrieved chunk → if not, return a clarification request instead of an answer. Reasoning: the grounding check is the guardrail that prevents confident fabrication, and the fallback keeps the agent from looping.
Traps
- Unbounded agent loops that burn tokens and never terminate.
- Hard-coding a model id or endpoint instead of externalising configuration.
- Resending the whole conversation every turn and silently exceeding the context window.
- No output validation before the response reaches a user or another system.