AI-300 flashcards

32 cards. Read the term and the definition here. The deck at the top of this page is optional.

Compute cluster

Autoscaling pool of multi-node virtual machines used for batch training and pipeline jobs; it scales down to zero when idle, unlike a compute instance used for interactive authoring.

Registry

Workspace-independent resource that stores and shares environments, components, and models across multiple workspaces in an organization, enabling reuse between teams and environments.

Managed online endpoint

Real-time deployment target that serves a model behind an HTTPS scoring URI with traffic splitting, autoscaling, and key or token authentication for callers.

Batch endpoint

Deployment target for long-running file-based scoring; each invocation creates a job that reads input from a datastore and writes predictions to an output path.

Job YAML

Declarative file beginning with type: command that names the code, environment, compute, inputs, and outputs, making the run reproducible from that file alone.

Environment versioning

Registering a Docker base image plus conda or pip dependencies under a name and version so training and scoring runs use identical, reproducible package sets.

Managed identity auth

Compute and workspace connections authenticate to storage, Key Vault, and model endpoints with a Microsoft Entra ID managed identity, removing secrets from code and configuration.

Model registry lineage

Each registered model version records the training job, data assets, and environment that produced it, so a deployment can be traced back to its exact inputs.

Blue-green deployment

Deploying the new model version to a second deployment at zero traffic, validating it, then shifting traffic gradually and rolling back by moving traffic back.

Data drift

Statistical change between the inference-time feature distribution and a reference baseline collected at training; monitored with metrics such as the population stability index.

Population stability index

Drift metric computed over binned distributions; values near zero mean no meaningful shift, while values above roughly 0.2 indicate a significant shift.

Autoscaling rule

Endpoint setting that adds or removes instances based on a metric such as CPU utilization or request latency, bounded by minimum and maximum instance counts.

Model card

Documentation shipped with a registered model describing intended use, training data, evaluation results, limitations, and fairness considerations for reviewers and downstream consumers.

Evaluation gate

Continuous integration step that compares the new model's metrics against thresholds on a fixed holdout set and blocks promotion when the comparison fails.

Foundry hub

Top-level Azure AI resource providing shared networking, connections, and policy for one or more projects, where each project contains its own models, flows, and indexes.

Prompt flow DAG

Directed graph of prompt, LLM, python, and tool nodes connected by named inputs and outputs, stored as YAML and executable locally or as a batch job.

Connection

Named, workspace-scoped record of an endpoint URL plus its credential, referenced by flows and deployments so no secret appears in prompt or code files.

Provisioned throughput

Model deployment option that reserves capacity in throughput units, giving predictable latency and cost for steady, high-volume generative AI workloads.

RAG ingestion pipeline

Ordered steps that load source documents, split them into overlapping chunks, compute embeddings, and write vectors plus metadata into a search index.

Trace

Record of a single request's node-by-node execution including prompts, retrieved context, token counts, and latency, exported through OpenTelemetry to monitoring.

Groundedness metric

AI-assisted score measuring whether every claim in the generated answer is supported by the supplied retrieved context, used to detect hallucination in RAG systems.

Golden dataset

Versioned, human-curated collection of representative questions with expected answers, reused across releases so evaluation scores remain directly comparable over time.

Retrieval recall

Share of relevant chunks that actually appear in the retrieved set; low recall means the model cannot produce a grounded answer regardless of prompt quality.

Red teaming

Structured adversarial testing with jailbreak, harmful-content, and prompt-injection inputs run as a batch before release, with findings converted into regression tests.

Severity scale

Content safety scoring from 0 to 7 grouped into four reporting bands (0, 2, 4, 6), where 0 is safe and 6 is high severity harm.

Continuous evaluation

Running AI-assisted metrics on a sampled fraction of live traffic and alerting when the rolling average falls below the release baseline.

Retrieval augmented generation

Pattern where relevant documents are retrieved and inserted into the prompt so the model answers from fresh, citable context instead of relying only on parametric memory.

Chunk overlap

Number of tokens shared between consecutive chunks so text spanning a boundary remains retrievable; typically ten to twenty percent of the chunk size.

Hybrid search

Combining vector similarity with keyword matching and an optional semantic reranker, improving recall for rare terms, product codes, and exact identifiers.

Semantic caching

Reusing a previously stored response when a new request is semantically similar enough to an earlier one, reducing both cost and latency.

Model routing

Sending simple queries to a smaller, cheaper model and complex queries to a larger model, balancing answer quality against cost and latency.

Prompt compression

Shortening instructions and trimming old conversation turns to reduce input tokens while preserving the system message and the retrieved context.