Deployment, monitoring, and lifecycle operations
Domain 7 - Deployment, monitoring, and lifecycle operations
Deployment questions test the path from an asset in a project to a monitored, versioned endpoint - and what you watch once it is live.
Deployment mechanics
A tuned model, prompt template or function is promoted from a project into a space and deployed with watsonx.ai Runtime. Online deployments serve low-latency real-time requests; batch deployments score large volumes on a schedule. You configure hardware sizing (including GPU) and auto-scaling, then score through the SDK (client.deployments.score) or REST.
What to monitor
- Operational: latency, throughput, error rate, availability.
- Cost: tokens consumed per request and per tenant.
- Quality: online evaluations, sampled human review, groundedness for RAG paths.
- Risk: drift against a baseline, HAP and PII incidents, prompt-injection attempts, fairness metrics.
- Business: task success and escalation rate.
Lifecycle
Version every artifact; keep a baseline evaluation set; promote through environments; roll back on regression; retire with a documented decommission step that updates the factsheet and removes access.
Worked example
A nightly job summarises 50,000 support tickets. Use a batch deployment with a scheduled job, not an online endpoint. Reasoning: throughput and cost matter more than sub-second latency, and online endpoints are billed for availability you do not need.
Traps
- No baseline, so drift cannot be detected.
- Monitoring infrastructure only, ignoring model quality decay.
- Changing a system prompt in production without re-running evaluations.
- Leaving an old deployment running after promotion, doubling cost.
- Assuming auto-scaling bypasses quota limits on the underlying model.
- Never decommissioning, so stale endpoints remain reachable.