AI-300 cheat sheet

The workspace is the top-level boundary for assets (datastores, environments, components, models); the registry is the cross-workspace sharing boundary.

Compute decision rule: compute cluster for batch training, compute instance for interactive authoring, managed online endpoint for real-time scoring, batch endpoint for file-based scoring, AKS only when full Kubernetes control is required.

Every asset is registered with a version, and jobs reference assets as name:version so a run is reproducible from its YAML alone.

A command job YAML declares type, environment, compute, code, command, inputs, and outputs; it is submitted with the az ml CLI or the Python SDK v2.

An environment is a base Docker image plus a conda or pip specification, versioned and registered; pin package versions so the image rebuilds identically.

Use managed identity for compute and for workspace connections so datastores, Key Vault, and model endpoints never require embedded keys.

Network isolation: private endpoints for workspace, storage, Key Vault, and registry; managed VNet with outbound rules, or VNet injection when an existing subnet must be used.

CI/CD sequence: lint and unit-test the scoring script, build and register the environment, submit the training pipeline, gate on evaluation metrics, then deploy via the same CLI in GitHub Actions or Azure DevOps.

Components must write only to the job's declared output paths, never to fixed storage locations, so repeated runs stay idempotent.

Model registration stores lineage back to the producing job, data assets, and environment; deployments reference model name:version rather than copied files.

Rollout rule: deploy the new version to a second deployment at 0% traffic, smoke test, shift 10% then 50% then 100%, and roll back by moving traffic back.

Data drift compares the inference distribution with a training or reference baseline; PSI near 0 means no shift and above roughly 0.2 indicates a significant shift.

Retraining triggers are a schedule, a drift threshold breach, or live-metric degradation; encode the trigger as a pipeline schedule or an event-driven job.

Evaluate on a fixed holdout using the same metric code that runs in production monitoring, and keep thresholds in CI rather than in notebooks.

Autoscale managed online endpoints on CPU utilization, memory, or request latency with a target utilization and bounded min/max instance counts.

Responsible AI deliverables before release: subgroup fairness metrics, cohort error analysis, feature importance, and a model card that ships with the registered model.

Promotion gate: the model moves to production only after the evaluation job passes thresholds and a recorded approval is stored in the model's tags or properties.

Hierarchy rule: hub holds shared networking, connections, and policy; each project holds its own deployments, flows, indexes, and evaluations.

Prompt flow is a DAG of prompt, LLM, python, and tool nodes linked by named inputs and outputs; the flow is YAML and can be executed as a batch job.

Connections store an endpoint plus credential under a name; flows reference the connection name so dev-to-prod promotion is a configuration change, not a code change.

RAG infrastructure chain: ingestion, chunking, embeddings deployment, search index with vector and keyword fields, retrieval node, grounded prompt, model call.

Deployment choice: standard for pay-per-token regional use, global standard for widest capacity, provisioned throughput for predictable latency under steady high volume.

Version in source control the prompt templates, flow YAML, index schema, evaluation datasets, and deployment names; deploy only through the pipeline.

Content safety and prompt shields are wired into the flow as nodes before the model call for input and after it for output.

Two metric families: heuristic (exact match, F1, BLEU, ROUGE) requires reference answers; AI-assisted (groundedness, relevance, coherence, fluency, similarity) requires a judge model.

Groundedness asks whether the answer is supported by the retrieved context; relevance asks whether the answer addresses the user question.

RAG-specific metrics: retrieval recall and precision on the returned chunks, plus answer completeness measured against a golden set.

The golden dataset is human-curated, representative of real traffic, and versioned, so scores from different releases are directly comparable.

Safety evaluators score harm categories such as hate, sexual, violence, and self-harm on a 0 to 7 severity scale, where 0 is safe and 6 is high.

Red teaming runs adversarial, jailbreak, and indirect prompt-injection prompts as a batch before release; each finding becomes a permanent regression test.

Production observability per request: trace of every node, input and output token counts, latency p50 and p95, error and refusal rate, user feedback, and sampled evaluation scores.

Continuous evaluation scores a sample of live traffic with AI-assisted metrics and alerts when the rolling mean drops below the release baseline.

Cost per request = (input tokens x input price + output tokens x output price) / 1000; output tokens usually cost more, so cap max output tokens.

RAG versus fine-tuning: RAG for fresh or citable knowledge, fine-tuning for tone, format, and task consistency, both together when both problems exist.

Chunking starting point is 256 to 512 tokens per chunk with 10 to 20 percent overlap, then tuned against retrieval recall on the golden set.

Retrieval tuning: hybrid vector plus keyword search with semantic reranking, then top-k of about 3 to 5 chunks; more context raises cost and can reduce groundedness.

Latency levers: streaming output, routing easy queries to a smaller model, provisioned throughput for steady load, and cached responses for repeated queries.

Prompt compression trims instructions and old conversation turns while keeping the system message and the retrieved context intact.

Structured output via JSON schema or tool calling reduces parse failures, retries, and downstream post-processing.

A/B rule: change one variable at a time, compare quality, latency, and cost on the same evaluation dataset, and keep the change only if it wins or the trade-off is explicitly accepted.