Implementing AI Solutions by Using Microsoft Foundry
The platform in one paragraph
Microsoft Foundry is the unified Azure platform for building AI apps: you create a Foundry resource/project, deploy models (including models from the model catalog), and consume them through the Foundry Models inference endpoint. Foundry also provides agent building, evaluation, tracing/observability, and connections to supporting Azure services. Generative apps typically use the Azure AI OpenAI-compatible chat completions API shape.
Concepts a candidate must know
- Deployment: a model must be deployed before you call it; the deployment has a name you reference in code, and a hosting option (for example standard pay-per-token versus provisioned throughput).
- Tokens: billing and context limits are measured in tokens, not characters. Prompts plus completion both consume the context window.
- Prompt engineering: system message sets behavior and constraints; user message carries the request. Few-shot examples, grounding data, and explicit output-format instructions improve reliability.
- Retrieval-augmented generation (RAG): retrieve relevant content from your own index or search store and inject it into the prompt so answers are grounded in your data rather than model memory.
- Agents and tools: an agent combines a model with instructions and callable tools (functions, search, code) to complete multi-step tasks.
- Evaluation: measure groundedness, relevance, coherence, and fluency; compare prompt variants before shipping.
- Content safety and responsible AI: filters and guardrails apply to inputs and outputs; know that configuration is part of the solution, not an afterthought.
Traps
- Calling a model by its catalog name when the code needs the deployment name.
- Assuming the model knows your private documents - without RAG or fine-tuning it does not.
- Forgetting that long conversations can exceed the context window; you must trim or summarize history.
- Confusing fine-tuning (changes model weights with your examples) with RAG (supplies context at query time).
- Ignoring cost drivers: output tokens usually cost more than input tokens.
Rule of thumb
If the knowledge changes often or must be cited, use RAG. If you need consistent style or task behavior across many calls, consider fine-tuning. If you need neither, a well-written system prompt is cheapest.
Worked example
A support bot must answer from a policy library updated weekly. Index the policies, retrieve the top matching passages per question, place them in the prompt, and instruct the model to answer only from that context and to say it does not know otherwise. This is RAG; re-indexing weekly keeps answers current without retraining.