watsonx GenAI Engineer flashcards

31 cards. Read the term and the definition here. The deck at the top of this page is optional.

Project versus deployment space

A project is a collaborative authoring container for notebooks, prompts, and model assets. A deployment space is the runtime container where promoted assets are deployed and where inference endpoints are created and managed.

watsonx.data lakehouse

An open lakehouse architecture that runs query and processing engines over object storage using open table formats, giving governed SQL access to data that watsonx.ai reaches through connection assets.

Context window

The maximum number of tokens a model can process in one call, counting the prompt plus generated output. Input and output tokens must together fit within this limit.

Temperature versus top_p

Temperature rescales the token probability distribution to control randomness; top_p restricts sampling to the smallest set of tokens whose cumulative probability reaches p. Adjust one at a time, not both.

Few-shot prompting

A prompting technique that includes several labeled input-output exemplars inside the prompt so the model infers the desired task and format without any weight updates.

System prompt

A persistent instruction message that sets a chat model's role, tone, and constraints for the whole conversation. It is sent on every call and consumes part of the context budget.

Chunk overlap

The number of tokens or characters shared between consecutive chunks during document splitting. Overlap preserves facts that straddle boundaries, commonly set to ten to twenty percent of chunk size.

Cosine similarity

A vector comparison measuring the cosine of the angle between two embeddings. It is standard for normalized text embeddings, where it ranks results identically to dot product.

Cross-encoder reranking

A second retrieval stage that scores each query-passage pair jointly with a cross-encoder. It improves precision over bi-encoder retrieval but adds latency, so it runs on a small candidate set.

Metadata filtering

Applying structured predicates such as source, date, or access-control tags during vector search. It narrows candidates and is the main mechanism for permission-aware retrieval in RAG.

LoRA tuning

Parameter-efficient fine-tuning that freezes base model weights and trains small low-rank adapter matrices. Trainable parameter count scales with the chosen rank rather than full model size.

Tuning versus RAG rule

Choose RAG when knowledge changes often or answers must cite sources. Choose tuning when the goal is style, output format, domain vocabulary, or behavior that prompts cannot reliably enforce.

Held-out validation split

Data reserved from tuning and used only for evaluation. It detects overfitting and data leakage, and it is the basis for claiming a tuned model beats its base model.

Disparate impact ratio

A fairness metric comparing favorable outcome rates between a protected group and a reference group. Values near one indicate parity; ratios below 0.8 are commonly flagged.

Model drift

A change over time in the distribution of production inputs or outputs relative to a baseline. Detection triggers review and possible retraining rather than automatic model replacement.

AI fact sheet

A governance artifact documenting a model's intended use, owners, training data references, limitations, and known risks. It is frequently requested by auditors and reviewers.

EU AI Act risk tiers

A regulatory classification into unacceptable, high, limited, and minimal risk. High-risk systems carry obligations for documentation, logging, human oversight, and conformity assessment.

LCEL chain composition

LangChain Expression Language composes steps such as retriever, prompt template, model, and output parser using pipe syntax, enabling streaming, batching, and async execution.

Tool calling pattern

The model emits a structured request naming a tool and arguments; application code executes it and returns the result as a new message so the model can continue reasoning.

Bounded conversation memory

A strategy keeping only a sliding window of recent turns or a rolling summary of older ones, preventing history from growing until it exceeds the context window.

Exponential backoff retry

A retry policy that increases wait time between attempts, with jitter, after throttling or server errors. It prevents synchronized retry storms against inference endpoints.

Online versus batch deployment

Online deployments serve synchronous real-time scoring requests through an endpoint. Batch deployments run scheduled scoring jobs over large datasets and write results to storage.

Deployment hardware sizing

The compute specification chosen when an online deployment is created. Changing it requires redeploying the model, so capacity must be planned before go-live.

Inference endpoint authentication

Calls to a deployed model require an IAM bearer token from a valid API key. Missing, expired, or mis-scoped credentials return 401 or 403 errors, not model errors.

Rollback strategy

Keeping the previous deployment live until the new one passes smoke tests, then shifting traffic. It restores a known-good model and prompt combination quickly.

Document ingestion pipeline

The sequence of connect, extract, clean, normalize, chunk, embed, and index with metadata. Each stage affects retrieval quality, and cleaning failures propagate into every answer.

Layout-aware PDF parsing

Extraction that preserves tables and multi-column reading order rather than naive text dumps. Scanned documents additionally require optical character recognition before chunking.

Ingest-time metadata capture

Recording source, section, page, timestamp, and access tags while documents are processed. Retrofitting metadata later requires reprocessing and re-embedding the entire corpus.

Full re-index trigger

Changing the embedding model or chunking scheme invalidates existing vectors because they are incompatible with new queries, so the whole index must be rebuilt.

Prompt injection via retrieval

Malicious instructions embedded in retrieved documents that the model may follow. Mitigation includes sanitizing retrieved text and instructing the model to treat context as data only.

Faithfulness metric

A generative evaluation metric measuring whether an answer's claims are supported by the retrieved context. Low faithfulness indicates hallucination despite relevant retrieval.