AI Solution Design and Deployment
What this domain tests
Whether you can move from a business decision to a working architecture - choosing serving patterns, meeting latency and cost targets, packaging the model, and releasing it safely.
Core rules
- Start from the decision and the consumer, not the model: what action the output triggers, who reads it, and what each error type costs.
- Capture non-functional requirements explicitly: p50/p95/p99 latency, throughput, availability, cost per inference, data residency, and auditability.
- Serving patterns: batch scoring for periodic bulk work, real-time APIs for interactive use, streaming for event pipelines, edge when connectivity or privacy forbids central inference.
- Build versus buy: a hosted foundation model with prompting or fine-tuning is often cheaper than a bespoke model, but check lock-in, data handling, and total cost of ownership including labelling and MLOps.
- Package reproducible artefacts: containers, a model registry, pinned dependency versions, a feature store, and infrastructure as code.
- Release with quality gates in CI, plus shadow, canary, or blue-green rollout and a tested rollback.
Worked example
A requirement of 150 ms at p95 against a 12-billion-parameter model that measures 900 ms. Reasoning: average latency is the wrong target; the tail drives user experience. Options in order of cost: quantise to int8, distill to a smaller student, cache frequent requests, batch offline traffic, or move non-interactive paths to batch scoring - accepting a small accuracy drop verified against the evaluation set.
Traps that cost marks
Designing to average latency instead of a tail percentile. Skipping the simple rules-based baseline. No rollback plan. Ignoring inference cost at production volume. Building without the owner of the downstream decision in the room.