DataRobot Certified Professional key terms

A multi-step process in which an AI agent plans, selects tools, executes actions, and evaluates results across several turns, rather than returning a single response to one prompt.

A mechanism that lets a language model invoke an external function, API, or data source by emitting structured arguments, with the platform executing the call and returning the result to the model.

The platform component that sequences agent steps, routes requests between models and tools, manages state, and enforces retries or fallbacks during a multi-step run.

A configured constraint that limits what an agent may do, such as blocking certain tools, capping spend, requiring approval before an action, or restricting topics the agent can address.

A checkpoint where an agent pauses and waits for a person to approve, edit, or reject a proposed action before the workflow continues to the next step.

A recorded sequence of an agent run's steps, including model calls, tool invocations, inputs, outputs, latency, and errors, used for debugging and audit.

A pattern where relevant documents are retrieved from a knowledge source and inserted into the model's context so answers are grounded in current, domain-specific content.

A store that indexes text or other content as numeric embeddings and returns the nearest matches to a query embedding, enabling semantic search over a knowledge base.

A dense numeric representation of text or other content produced by a model, where items with similar meaning land close together in vector space.

The practice of splitting source documents into smaller passages before indexing, balancing passage size so retrieved context is specific enough yet still contains complete ideas.

Tying a generated answer to supplied evidence such as retrieved passages or tool outputs, so claims can be checked against sources rather than relying on model memory.

A generated statement that is fluent but unsupported or factually wrong, arising when a model fills gaps from learned patterns instead of from reliable evidence.

An attack where untrusted content, such as a retrieved page or user message, contains instructions that try to override the system prompt or redirect the model's behavior.

A generation quality measure of whether every claim in an answer is supported by the provided context, as opposed to relevance, which concerns whether the answer addresses the question.

Automated profiling of a dataset that identifies candidate input variables, their types and distributions, and flags leakage or quality problems before modeling begins.

A situation where an input feature contains information that would not be available at prediction time, or directly encodes the outcome, producing optimistic scores that fail in production.

A predefined modeling recipe that pairs a data preparation pipeline with one or more algorithms and their default settings, used as a starting point in automated model selection.

The automated mode that runs feature engineering, model training, and tuning across many candidate approaches, then ranks results so a user can choose a model to deploy.

A ranked comparison of trained models scored on a chosen validation metric, letting a practitioner weigh accuracy against training time and interpretability.

Data withheld from training and used once to estimate how a selected model will perform on unseen cases, guarding against overfitting to the validation data.

A resampling procedure that splits training data into folds, trains on some folds and scores on the rest repeatedly, producing a more stable performance estimate.

A model-agnostic explanation showing how much each input changes predictions on average, typically computed by perturbing a feature and measuring the shift in output.

Per-row reasons showing which input values pushed a single prediction up or down and by how much, used to justify decisions to reviewers, customers, or regulators.

A catalog that tracks trained model versions, their metadata, training data lineage, and deployment status, so teams can promote, roll back, and audit models.

A structured record describing a model's purpose, data, performance, limitations, and monitoring plan, often assembled to satisfy internal review and regulatory expectations.

Evaluating model outcomes across protected or sensitive groups to detect disparate error rates or selection rates, and deciding whether mitigation or rejection is warranted.

A quantitative measure of outcome parity between groups, such as equal error rates or similar selection rates; different metrics can conflict, so the choice must match the use case.

A change over time in the distribution of input features compared with the training data, which can degrade predictions even when the underlying relationship is unchanged.

A change in the relationship between inputs and the target, so the same input values now imply different outcomes and the model's learned mapping becomes stale.

Ongoing comparison of live input and prediction distributions against a baseline, with thresholds and alerts that flag when a model needs retraining or investigation.

Tracking operational signals of a deployed model endpoint, such as request volume, latency, error rates, and availability, to catch failures that are not statistical drift.

A deployment pattern where the current production model serves traffic while one or more challenger models score the same requests, so their performance can be compared safely.

Instrumentation of generative applications that records prompts, responses, token usage, latency, and quality scores so teams can detect regressions and control cost.

A score expressing how close two pieces of text are in meaning, usually computed from embeddings, and used to rank retrieved passages or compare generated answers.

Deliberate adversarial probing of a model or agent with hostile prompts and scenarios to surface unsafe behavior, data leakage, or guardrail gaps before release.

Creating or transforming input variables, such as ratios, date parts, or text encodings, to make patterns easier for a model to learn and improve predictive performance.