Large Language Models and Prompt Engineering in watsonx Assistant
What this domain tests
This domain tests practical LLM use inside watsonx Assistant: model choice, prompt structure, variables, grounding, output control and evaluation. It is not about training a foundation model.
Core rules
- Use an LLM when the task is generative or ambiguous; use deterministic steps for exact lookups, calculations and policy checks.
- A prompt step typically combines a system instruction, task instruction, context variables and the user input. Delimit user content clearly so it cannot override instructions.
- Ground factual answers in retrieved passages or tool results. Instruct the model to use only the supplied context and to say when the context is insufficient.
- Control output: specify format, length, tone and allowed values. Lower temperature for factual or repeatable tasks; raise it only for brainstorming.
- Respect token limits. Retrieve less but more relevant context rather than stuffing the window.
- Version prompts and test them against a labeled set. Model or prompt changes can alter behavior, so re-run regression cases.
Worked example
Task: summarize a support ticket into three bullets for an agent. System: “You summarize only from the ticket text; do not invent causes.” User template includes ticket_text and instruction: “Return exactly three bullets: issue, customer impact, requested action.” Temperature 0.2, max tokens 180. If ticket_text is empty, the action says no summary is available rather than calling the model. Reasoning: the system prompt constrains invention, the template fixes format, low temperature improves consistency, and the empty-input guard prevents a hallucinated summary.
Traps
Putting secrets or full user records in prompts unnecessarily; allowing user text to close delimiters; using high temperature for policy answers; assuming a prompt is a security boundary; and failing to re-test after a model upgrade. Another trap is asking the model for a number that a deterministic step should compute.