Databricks GenAI Engineer cheat sheet

Foundation Model APIs bill per token; provisioned throughput bills per hour and reserves capacity for latency-sensitive workloads.

Serving endpoints expose an OpenAI-compatible /chat/completions route; set base_url to https:///serving-endpoints and authenticate with a Databricks token.

WorkspaceClient().serving_endpoints.get_open_ai_client() returns an OpenAI client preconfigured for the workspace.

LangChain integration uses ChatDatabricks from databricks-langchain, passing the endpoint name and optional temperature and max_tokens.

Custom models deploy via MLflow pyfunc or logged transformers models; GPU endpoints are needed for large models, CPU for small classifiers.

RAG decision rule: retrieve top-k chunks (commonly 3-10), insert them into the prompt, and instruct the model to answer only from the provided context to reduce hallucination.

Streaming responses use stream=True; tokens arrive as server-sent events, reducing perceived latency for chat interfaces.

Agent tools are Python functions described with docstrings; the LLM selects tools by name and arguments, so precise descriptions improve tool selection.

Model Serving supports scale-to-zero, concurrency, and autoscaling; choose provisioned throughput when pay-per-token latency is too variable.

A Delta Sync index automatically updates from a Delta source table; Direct Vector Access requires manual upserts and deletes.

Vector Search requires a primary key column, an embedding column of type array, and optional metadata columns for filtering.

The embedding model used at index time must match the model used at query time; changing models requires re-embedding and reindexing.

Chunk size is commonly 200-1000 tokens with 10-20% overlap; smaller chunks improve retrieval precision, larger chunks preserve context.

Use delta readChangeFeed or a streaming table to incrementally embed only new or changed rows rather than recomputing the full corpus.

ai_parse_document extracts text from PDFs and images; normalize, deduplicate, and remove boilerplate before chunking.

Batch embedding runs with ai_query or pandas UDFs; persist embeddings to Delta so downstream indexes read a single source of truth.

Filter metadata columns at query time to constrain retrieval by source, department, or date, improving relevance and supporting access control.

Built-in embedding endpoints include databricks-bge-large-en; dimension mismatch with the index configuration causes upsert failures.

mlflow.evaluate with model_type='databricks-agent' runs built-in scorers including relevance, groundedness, safety, and retrieval metrics.

LLM-as-judge metrics are defined with make_genai_metric, specifying a judge model, a rubric, and examples.

Evaluation datasets pair inputs with optional expectations; store them in Delta for versioned, reproducible comparisons.

Fine-tuning on Foundation Model APIs uses instruction/response JSONL; supported families include Llama and MPT, and LoRA reduces trainable parameters.

MLflow tracing records each agent step (retrieval, tool call, LLM call) so failures can be attributed to a specific stage.

Prompt patterns: a system prompt sets role and constraints; few-shot examples improve format adherence; chain-of-thought improves multi-step reasoning.

Compare runs in MLflow only when the same metric names and dataset version are logged, because scores are otherwise not comparable.

Human feedback collected in the Review App becomes labeled data for evaluation or fine-tuning.

Enable inference tables on a serving endpoint to log request and response payloads to a Delta table.

Traffic splitting routes a percentage of requests to each served model version; use it for canary releases and A/B tests.

Provisioned throughput endpoints reserve capacity; use them when p95 latency or throughput must be guaranteed.

Lakehouse Monitoring profiles inference tables for drift, data quality, and custom metrics; the refresh schedule controls cost.

system.serving and system.billing.usage system tables expose endpoint usage and cost for chargeback and alerting.

Databricks Asset Bundles deploy jobs, pipelines, and endpoints via databricks bundle deploy with environment-specific targets.

Scale-to-zero reduces idle cost but adds cold-start latency; keep min replicas greater than zero for latency-sensitive endpoints.

Alert on monitoring metrics via SQL alerts or scheduled jobs that query the monitoring output tables.

Unity Catalog governs models, functions, and endpoints; GRANT EXECUTE ON FUNCTION and GRANT SELECT on tables enforce least privilege.

AI Gateway provides rate limits, payload logging, usage tracking, and guardrails such as PII detection and content safety on serving endpoints.

Store API keys and tokens in Databricks secrets and reference them with dbutils.secrets.get rather than hardcoding values.

system.access.audit records access and changes; join it with system.billing.usage for cost attribution by user.

PII redaction can use ai_mask or presidio before logging prompts and responses to inference tables.

Responsible AI evaluation includes safety, toxicity, and bias scorers in mlflow.evaluate alongside quality metrics.

Service principals with OAuth authenticate applications; grant only the catalogs, tables, and endpoints the application needs.

Model lineage in Unity Catalog tracks which data and notebooks produced a model version, supporting audits and reproducibility.