Model Development, Tuning, and Deployment in watsonx.ai
This domain tests notebook workflow, runtime environments, tuning options and the deployment types available in watsonx.ai.
Notebooks and environments
JupyterLab notebooks run against runtime environments (Python, R, Spark, GPU) defined by software specifications. Assets saved to the project persist; the kernel and its in-memory state do not. Promote assets to a deployment space to serve them.
Tuning choices
- Prompt tuning - learns soft prompt vectors; base weights unchanged; suits small data and a small GPU budget.
- Fine-tuning - updates model weights on labeled data; needs more data and compute.
- QLoRA - quantized base plus low-rank adapters; parameter-efficient and cheap.
Deployment types
Online deployment yields a REST scoring endpoint with optional payload logging; batch deployment scores a data asset as a scheduled job. Foundation models can be deployed for inferencing on supported hardware profiles, and prompt templates can be saved as deployable assets.
Worked example
8,000 labeled support tickets, one mid-range GPU, and unfamiliar domain vocabulary. Reasoning: full fine-tuning is likely infeasible at that budget, so QLoRA adapts the model cheaply. Deploy online, send a test payload, compare against the base model on a held-out set, then promote.
Traps
Assuming prompt tuning changes weights; forgetting that assets must reach a deployment space; runtime environment version mismatch between notebook and deployment; skipping a smoke test after deploy; tuning on the same data used for evaluation.