AI-103 common mistakes
AI-103 · Understand the traps and how to avoid them.
Treats API keys as the default authentication path for every Azure AI resource and then misassigns RBAC roles when asked to switch to keyless auth - for example granting Cognitive Services Contributor and expecting the app to call the inference endpoint, or granting Cognitive Services User and expecting it to call an Azure OpenAI model deployment.
Candidates memorise 'use managed identity' as a slogan without separating control-plane roles (create/manage resources) from data-plane roles (invoke models and analyzers), so they pick a role that reads correctly but authorises the wrong operation.
What to do instead
Before answering, classify the action as management (Contributor/Owner on the resource) or inference (data-plane roles such as Cognitive Services OpenAI User, Cognitive Services User, Search Index Data Reader/Contributor), then verify the identity has a role assignment scoped to the specific resource and that local auth (key access) can be disabled without breaking the caller.
Plans a model deployment without checking regional model availability, quota headroom, or deployment type, then designs a solution around a Standard deployment when the workload's latency/throughput requirement clearly needs Provisioned Throughput Units or a different region.
The planning questions look like pure architecture questions, so candidates answer with the 'best' service tier rather than reasoning about the concrete constraints (tokens-per-minute quota, region model catalog, capacity units) that decide whether a design is even deployable.
What to do instead
For any deployment scenario, state three checks explicitly: is the model version in the target region's catalog, what TPM/RPM quota exists and is it enough for peak load, and which deployment type matches the load profile (Standard/pay-as-you-go for spiky low volume, Provisioned for predictable high volume, batch for offline bulk). If a constraint fails, propose the region change or quota increase as part of the answer.
Recommends fine-tuning a model to make it answer questions about the company's internal policies and product catalogue, instead of grounding the model on a search index at query time.
Fine-tuning and retrieval both 'add knowledge' in casual conversation, so candidates default to the more memorable feature and miss that fine-tuning teaches form and behaviour, not reliably retrievable facts.
What to do instead
Use the decision rule: facts that change and must be cited → retrieval grounding (index + citations); consistent tone, format, or task behaviour and labelled example pairs → fine-tuning; both → fine-tune for behaviour, ground for facts. Say in the answer that fine-tuning does not give you citations and cannot guarantee freshness.
Designs an agent or function-calling flow as a single request/response pair - sends the user message with tool definitions and expects the model to return the final answer, never sending the tool result back to the model.
The tool call is visually a normal completion response, so candidates assume the service executes the tool. The orchestration loop (model returns tool call → application executes → tool output appended as a tool message → model called again) is the part that is easy to skip when reading rather than building.
What to do instead
When describing any agent or function-calling flow, draw the loop: receive tool call, validate the arguments against the schema, execute the function, append the result with the correct role, re-invoke the model, and cap iterations. Also state that the tool JSON schema (names, required parameters, descriptions) is what the model uses to choose the tool, so vague descriptions cause wrong tool selection.
Ships a generative solution design with no evaluation plan and no safety configuration - treats Foundry's evaluation metrics and content filters as optional extras rather than part of the architecture.
Evaluation and safety feel like operational afterthoughts compared with the exciting parts (models, agents, tools), so candidates describe the happy path and lose the marks attached to measuring groundedness/relevance and configuring harm categories.
What to do instead
Include a named evaluation step (a test dataset plus groundedness, relevance, coherence, and fluency evaluators, run before and after changes) and a named safety step (content filter severity thresholds per harm category, prompt shields for jailbreak/indirect injection, and human review for high-risk outputs) in every generative design answer.
Picks the wrong vision capability for the stated goal - uses image classification when the requirement is to locate and count items, uses a general image-analysis captioning model when the requirement is to extract structured fields from scanned forms, or reaches for Custom Vision when the built-in Image Analysis features already satisfy the requirement.
All vision tasks feel like 'look at a picture and tell me something', so candidates map on surface vocabulary rather than on output shape: one label per image (classification), labels plus bounding boxes (object detection), text transcription (Read/OCR), key-value pairs from documents (Document Intelligence), or open-ended description (captioning/dense captions).
What to do instead
Choose by output shape and data, in this order: is the answer a per-image class (classification), a location (object detection), raw text (Read/OCR), structured fields from a form or invoice (Document Intelligence prebuilt/custom), or a free-text description (image analysis captioning)? Only then decide whether a prebuilt model is sufficient or a custom-trained model is required.
Ignores the input constraints and failure responses of vision APIs - assumes any image works, does not check for unsupported formats, oversized files, or low-resolution text, and does not handle region/deployment availability or limited-access services before they appear in the design.
Sample code always uses a clean, small JPEG, so the practical constraints (size/dimension limits, minimum text height for reliable OCR, per-region feature availability, gated features) are invisible until the candidate is asked what happens with real-world inputs.
What to do instead
In any vision design, add an explicit preprocessing and error-handling layer: validate format/size, downscale or split large images, apply orientation correction, check minimum text height for OCR, set and use a confidence threshold for downstream decisions, and confirm the feature is available in the chosen region and that gated capabilities are approved for the subscription.
Confuses the Azure Language project types - builds a conversational question-answering project (CQA) for a task that needs intent classification with slot filling, or builds a custom named entity extraction project for a task that needs intent routing - and then wires the wrong orchestration between them.
Conversational language understanding, custom question answering, and orchestration all consume 'user utterances' and all produce 'an answer', so the distinguishing signal (does the app need to route intent and fill parameters, or retrieve an answer from a knowledge base?) is lost. The orchestrator project type that routes between them is also frequently forgotten.
What to do instead
Map the requirement first: route user utterances to actions and extract parameters → conversational language understanding; retrieve answers from an FAQ/document knowledge base → custom question answering; both plus a generative fallback → orchestration workflow that dispatches to each. Then state which resource and project type each component lives in and how confidence thresholds drive fallback.
Selects the wrong Document Intelligence model family for a custom extraction task - trains a custom template model on documents with variable layouts, or builds separate single-purpose models where a composed model is required - and does not distinguish prebuilt, custom template, custom neural, and composed models.
All custom models are trained from labelled samples and return fields, so the decisive differences (fixed layout vs variable layout, structured vs unstructured, labelling effort, per-document routing) only matter when the candidate is forced to reason about document variety rather than about the API call.
What to do instead
Choose by document variety: fixed, stable layout → custom template model; variable or unstructured layout (contracts, letters) → custom neural model; multiple known document types in one submission → train per type and compose them so the composed model routes automatically; standard invoice/receipt/ID/W-2 style documents → check prebuilt models first. State the training data labelling approach (layout files with labelled fields) and how confidence scores gate human review.
Designs a retrieval index for a RAG pipeline without matching the chunking strategy, embedding model dimension, and field attributes to the query pattern - mismatched vector dimensions, no filterable fields for security trimming, and no hybrid or semantic ranking.
Index design looks like plumbing compared with the model choice, so candidates list 'create a search index' without specifying chunk size/overlap, the embedding deployment used at both indexing and query time, field attributes, and the retrieval mode, which is exactly what determines answer quality.
What to do instead
Specify the full contract: chunk size and overlap tied to the content type, the same embedding model deployment used for indexing and querying with a vector field dimension that matches the model, searchable/filterable/retrievable attributes set per field, a filterable security or tenant field for trimming, and hybrid search (keyword + vector) with semantic ranking enabled for relevance.