Databricks GenAI Engineer key terms

A pattern where an application searches a knowledge base for relevant passages and inserts them into the prompt, so the model answers from grounded, current evidence instead of relying only on training data.

A large pretrained model, such as a Llama or DBRX variant, that can be adapted to many tasks through prompting or fine-tuning instead of being trained from scratch for one purpose.

Databricks-hosted endpoints exposing pretrained chat, completion, and embedding models, billed per token or through reserved capacity, letting teams prototype without managing GPU infrastructure themselves.

The maximum number of tokens a model can process in one request, covering the system prompt, user input, retrieved passages, and the generated answer combined.

Instructions placed ahead of user input that set the model's role, tone, constraints, and output format, persisting across turns and steering behavior for the whole conversation.

A reusable parameterized string that combines fixed instructions with runtime variables such as retrieved context and the user question, keeping prompts consistent and testable across an application.

A model capability where the LLM returns a structured request naming a tool and its arguments, letting the application run code, query data, or call APIs before continuing the conversation.

An LLM-driven program that plans, selects tools, executes steps, and observes results in a loop until a user request is satisfied, rather than producing a single response.

A model that converts text into vectors; the same model must encode both indexed documents and incoming queries, otherwise similarity scores become meaningless.

A Databricks-managed similarity index that syncs automatically from a Delta table with change data feed, embedding source text and serving nearest-neighbor queries for retrieval.

A dense numeric vector representing text meaning, so semantically similar passages land close together and can be compared using distance or similarity metrics.

Splitting source documents into smaller passages before embedding, balancing chunk size against retrieval precision and the amount of context that will fit in the prompt.

Converting PDFs, HTML, and other files into clean structured text, preserving headings and tables so each chunk carries accurate meaning before it is embedded.

Combining vector similarity with keyword matching such as BM25 and merging the ranked results, which improves recall for exact identifiers, product codes, and rare terms.

A second-stage model that rescores a shortlist of retrieved passages for relevance to the query, improving the ordering of context sent into the prompt.

Restricting a vector search to rows matching conditions on fields such as department, date, or access level, keeping results both relevant and permitted for the requesting user.

An open table format on cloud storage adding transactions, schema enforcement, and time travel, serving as the governed source of truth that RAG indexes sync from.

Further training a pretrained model on your own labeled examples to change its behavior, style, or domain accuracy; it teaches skills and formats rather than adding fresh factual knowledge.

A parameter-efficient technique that freezes the base model and trains small low-rank matrices, cutting GPU memory and cost while still adapting style and task behavior.

An open-source platform for tracking experiments, logging parameters and metrics, versioning models in a registry, and packaging them for deployment across the lifecycle.

Structured logging of every step in a generative AI request, including retrieval, prompt assembly, model calls, and tool use, so developers can debug quality issues and inspect traffic.

An evaluation method where a capable model scores another model's responses against a rubric, scaling subjective quality checks that would be costly to run with human reviewers.

The degree to which an answer's claims are supported by the retrieved context, a key quality metric for detecting unsupported statements in retrieval augmented systems.

A confident model output that is factually wrong or unsupported by the supplied context, often caused by retrieval gaps or prompts that invite guessing.

A curated set of questions paired with expected answers or reference facts, used to score retrieval and generation quality before and after changes.

A sampling parameter that scales randomness in token selection: low values make output focused and repeatable, while higher values increase variety at the cost of consistency.

A decoding setting that restricts choices to the smallest set of tokens whose cumulative probability reaches p, another lever for controlling creativity alongside temperature.

A catalog for versioning trained and fine-tuned models, recording lineage and approval status, and promoting a chosen version to a serving endpoint.

A Databricks-managed REST API that hosts a model or chain, handling scaling, authentication, and rollout so applications can call it over HTTPS.

Reserving dedicated model capacity for predictable latency and higher rate limits, billed by the hour rather than per token, typically for production workloads.

Delta tables that automatically capture requests and responses sent to a serving endpoint, enabling monitoring, debugging, and quality evaluation of live traffic.

A Databricks feature that builds profile and drift metrics over Delta tables, including inference tables, and raises alerts when data or model behavior shifts.

Routing a percentage of requests to a new model version while the remainder continue to the current one, supporting canary releases and A/B comparisons before full cutover.

A unified governance layer for data and models that manages permissions, lineage, auditing, and discovery across workspaces and cloud storage.

A governance layer in front of model endpoints that logs payloads, enforces rate limits, filters unsafe content, and applies usage policies consistently across applications.

Input and output filters that block prompt injection, unsafe content, or disallowed topics, typically implemented as policy checks before and after the model call.

Detecting and masking personal identifiers in prompts, source documents, or request logs so sensitive data is not sent to a model or retained in monitoring tables.

Records of who accessed or changed data, models, and endpoints, used to investigate incidents and demonstrate compliance during security and privacy reviews.

Granting or denying permissions on tables, models, and endpoints through catalog privileges, so users and service principals see only the data their role allows.

An attack where untrusted text in a document or user message tries to override system instructions, causing the model to leak data or take unintended actions.