Implement Generative AI Solutions
Core objects
In Azure OpenAI, you create a resource, then deploy a model. The deployment name is what code sends as model; the underlying model name may differ, and this is the single most common exam trap. Calls go to an endpoint with either a key or an Entra ID token, and the chat API takes an ordered list of messages with roles system, user, assistant.
Controlling output
temperature and top_p trade determinism for creativity; max_tokens caps the completion, not the whole context window; stop sequences, seed and response_format: json shape results. Know that prompts plus completion must fit the context window, and that long chats must be trimmed or summarised.
Quotas and reliability
Throughput is metered in tokens/requests per minute per deployment. Exceeding it returns HTTP 429; the fix is exponential backoff with jitter, spreading load across several deployments behind a gateway, or provisioned throughput when latency must be predictable.
Grounding and customisation
Retrieval-augmented generation: chunk documents, embed them, index in Azure AI Search, retrieve top-k, and inject the passages into the prompt with instructions to cite and to answer only from context. Fine-tuning (supervised chat in JSONL, plus preference/DPO tuning) teaches style and format, not new facts. Content filters apply to prompt and completion across harm categories with severity thresholds; blocklists catch custom terms.
Evaluation
Measure groundedness, relevance, coherence and fluency with the evaluation SDK and human review.
Traps and rule of thumb
Don't confuse model name with deployment name, don't assume fine-tuning adds knowledge, and don't ignore token budgeting. Rule of thumb: facts come from retrieval, behaviour comes from prompting or fine-tuning.