OCI GenAI Pro flashcards

32 cards. Read the term and the definition here. The deck at the top of this page is optional.

Token

The subword unit a language model reads and generates. OCI Generative AI measures context limits and on-demand billing in tokens, so prompt length and output length compete for the same budget.

Temperature

A sampling parameter that scales randomness in token selection. Near zero yields focused, repeatable output; higher values increase diversity and the risk of drifting from the instruction.

Top-p sampling

Nucleus sampling that keeps the smallest set of tokens whose cumulative probability reaches p, then samples from that set. It truncates the long tail of unlikely tokens.

Context window

The maximum number of tokens a model can attend to in one request, covering prompt plus generated output. Exceeding it requires truncation, chunking, or summarization.

Embedding

A dense numeric vector representing text meaning. Embeddings enable semantic search and clustering; similarity between vectors is measured with cosine or dot product.

Cosine similarity

A similarity measure equal to the dot product of two normalized vectors, ranging from -1 to 1. It ignores vector magnitude, so it suits comparing embeddings of differing lengths.

Hallucination

Fluent model output that is not supported by provided context or facts. Retrieval grounding and output validation reduce it, but no prompt fully eliminates it.

On-demand inference

Shared-capacity inference billed per token with no cluster provisioning. It suits variable workloads and pretrained models, but cannot host fine-tuned custom models.

Dedicated AI cluster

Reserved OCI compute that hosts model endpoints with predictable throughput. It is required for serving fine-tuned models and for workloads needing guaranteed capacity.

Model size trade-off

Larger model variants generally improve quality while increasing latency and per-token cost. Choose the smallest variant that passes your evaluation set for the task.

Fine-tuning

Parameter-efficient training that adapts a base model to a domain, tone, or output format using labeled examples. It shapes behavior rather than injecting frequently changing facts.

RAG versus fine-tuning

Use retrieval when facts change often or need citations; use fine-tuning when the gap is style, format, or task behavior. They are complementary and often combined.

Streaming inference

Returning generated tokens incrementally as they are produced. It improves perceived responsiveness without changing total generation time or throughput capacity.

Few-shot prompting

Including a small number of worked input-output examples in the prompt to demonstrate the desired format or reasoning pattern. It requires no training and no model changes.

System prompt

Instruction text that sets the model's role, constraints, and output format ahead of user content. It frames behavior but is not a security boundary against prompt injection.

max_tokens parameter

A cap on generated output length. It does not limit prompt size, and prompt tokens plus this cap must fit inside the model's context window.

Chunking

Splitting source documents into passages small enough for the embedding model and coherent enough to stand alone. Overlap between chunks preserves context at boundaries.

Top-k retrieval

Returning the k nearest vectors to a query embedding. Raising k improves recall but adds prompt tokens and can dilute relevance with weaker passages.

Reranking

A second-stage model that rescores retrieved candidates for relevance to the query. It improves precision when retrieving many candidates before selecting the final few.

Hybrid search

Combining keyword matching such as BM25 with dense vector similarity. Keyword handles exact identifiers and rare terms that embeddings tend to blur.

OCI Search with OpenSearch

A managed OpenSearch service in OCI supporting vector fields and k-nearest-neighbor queries. It commonly serves as the vector store backing retrieval-augmented generation.

Grounding prompt

A prompt that supplies retrieved passages in a delimited block and instructs the model to answer only from them, with a defined fallback when context is insufficient.

Retrieval recall

The share of queries whose correct answer appears in the retrieved chunks. Recall failures cap end-to-end RAG quality regardless of prompt or model choice.

Metadata filtering

Applying attribute filters such as tenant, date, or classification during vector search. It narrows results and enforces authorization so retrieval cannot cross access boundaries.

IAM policy scope

OCI policy statements naming compartments, resource types, and verbs. Least privilege means scoping generative AI access narrowly instead of granting tenancy-wide family permissions.

Instance principal

An identity that lets a compute instance call OCI APIs using its own credentials rather than a user's. Policies must still explicitly grant the required access.

OCI Vault

A managed secrets service for storing API keys and credentials. Applications retrieve secrets at runtime instead of embedding them in code or prompt templates.

Audit versus logging

Audit records control-plane API activity, while logging and monitoring capture application request and response telemetry. Governance needs both plus a retention policy.

Prompt injection

An attack where untrusted text, often retrieved or user-supplied, contains instructions that hijack model behavior. Treat such content as data and isolate it with delimiters.

Compartment design

OCI compartments separate environments, teams, and cost centers. Policies and budgets scope to compartments, so layout determines both isolation and chargeback granularity.

Graceful degradation

A designed fallback when model capacity or quota is exhausted, such as a smaller model, cached answer, or queued retry, so the application fails partially rather than fully.

Private endpoint access

Reaching OCI services through a private network path instead of the public internet. It reduces exposure for sensitive prompts and is a common compliance requirement.