Foundation Models and Prompt Engineering in watsonx.ai
This domain tests model selection in the watsonx.ai catalog, the Prompt Lab controls, and the prompt patterns that make output reliable.
Catalog and limits
watsonx.ai exposes a curated catalog (Granite family plus third-party open models), each with a model card describing training data, intended use, languages and context window. Prompt tokens plus generated tokens must fit that context window.
Controls that change output
- temperature - sampling randomness; 0 is nearly deterministic.
- top_p / top_k - truncate the token distribution (nucleus vs fixed count).
- max_new_tokens - caps output length only.
- stop sequences - end generation at a marker.
- repetition penalty - discourages loops.
Prompt patterns
Zero-shot instruction; few-shot with labeled exemplars; chain-of-thought for multi-step reasoning; a role or system prompt for behavior; delimiters and an explicit output schema.
Worked example
An extraction prompt sometimes returns prose, sometimes JSON. Reasoning: set temperature to 0 for determinism, add two few-shot examples in the exact target JSON shape, state "return only JSON", and add a stop sequence; validate the shape downstream. Changing only the model rarely fixes format instability.
Traps
Treating temperature and top_p as the same knob; confusing max tokens with the context window; exemplars with inconsistent formatting; assuming a larger model beats a well-structured prompt; putting instructions after the data they govern.