Claude Certified Architect - Foundations key terms

The maximum number of tokens a model can consider at once, covering system prompt, conversation history, retrieved documents, and the response. Inputs beyond this limit must be truncated, summarized, or split across calls.

The basic unit of text a model reads and generates, roughly a word fragment. Token counts drive cost, latency, and context limits, so estimating them accurately matters for capacity planning.

A per-request cap on how many tokens the model may generate. Hitting the cap truncates the response, so long-form tasks need either a higher limit or chunked generation.

A sampling parameter controlling randomness. Lower values make output more deterministic and focused; higher values increase variety. Extraction and classification tasks usually favor low values.

Nucleus sampling that restricts generation to the smallest set of tokens whose cumulative probability exceeds a threshold. Adjusting it together with temperature changes output diversity.

A string that halts generation when produced. Useful for delimiting structured output or ending a turn early, but the stop text itself is typically excluded from the returned content.

The assistant turn partially provided by the caller so the model continues from it. It constrains output format or role, and it cannot be used to force content the model would otherwise refuse.

Instructions supplied separately from user messages that set role, tone, constraints, and available tools. It carries higher priority than user turns and should be explicit about refusals and output format.

The ordering in which system instructions outrank user messages, which outrank content inside tool results or retrieved documents. Attacks succeed when untrusted text is treated as instructions rather than data.

Including several worked examples in the prompt so the model can imitate the desired pattern. Examples should be varied, correctly labeled, and representative of edge cases you expect.

Asking the model to reason step by step before answering, which often improves multi-step accuracy. The reasoning can be hidden from end users while the final answer is surfaced.

Constraining responses to a schema such as JSON so downstream code can parse them reliably. It usually combines explicit format instructions, examples, and validation with retry on failure.

A reusable prompt with placeholders filled at runtime. Good templates separate static instructions from variable data and guard against untrusted values altering the instructions.

An attack where text in user input, documents, or tool output tries to override the developer's instructions. Defenses include instruction hierarchy, delimiting untrusted content, and least-privilege tools.

Injection delivered through content the model processes rather than the user's direct message, such as a retrieved web page or file. It is dangerous because the attacker never talks to the app directly.

Giving the model a declared set of functions it can request, with the application executing them and returning results. The model chooses arguments; your code decides whether and how to run them.

The name, description, and JSON input definition that tells the model when and how to call a tool. Clear descriptions and tight parameter types reduce malformed or unnecessary calls.

The output returned to the model after a tool executes, typically as a user-role content block. It is untrusted data and should never be interpreted as higher-priority instructions.

The cycle of model reasoning, tool call, result observation, and repeat until a stopping condition. Production loops need iteration caps, timeouts, and cost limits to prevent runaway behavior.

The application layer that manages model calls, routes tool requests, tracks state, and enforces limits. It, not the model, owns control flow and side effects.

Requiring human approval before high-impact actions such as payments, deletions, or external messages. It is a primary control for agent autonomy where errors are costly or irreversible.

Granting a tool or agent only the permissions needed for its task. Scoping credentials and restricting filesystem or network access limits damage from mistakes or injected instructions.

Running generated code or tool actions in an isolated environment with restricted resources and no sensitive network access. It contains failures and malicious behavior during execution.

Fetching relevant documents at query time and placing them in the prompt so answers are grounded in current, private knowledge rather than only training data.

A numeric vector representing text meaning, used to find semantically similar passages. Similarity is usually measured with cosine distance between query and document vectors.

Splitting source documents into passages small enough to embed and fit in context. Chunk size and overlap trade retrieval precision against losing surrounding meaning.

A store that indexes embeddings and returns nearest neighbors for a query vector. It enables semantic search over large corpora without scanning every document.

Retrieval ranked by meaning similarity rather than exact keyword match. It handles paraphrase well but can miss rare identifiers, so hybrid approaches with keyword search are common.

Combining lexical keyword search with semantic vector search and merging the ranked results. It improves recall for exact terms such as product codes while keeping meaning-based matching.

A second-stage model that reorders initially retrieved passages by relevance to the query. It improves precision when the first-stage recall set is broad.

Tying model answers to supplied sources so claims can be traced and verified. Citations and explicit instructions to answer only from provided context reduce fabrication.

Confidently stated output that is unsupported or false. Mitigations include grounding in retrieved sources, asking the model to abstain when evidence is missing, and automated verification.

A control that constrains model behavior, such as input filtering, output validation, topic restrictions, or refusal rules. Guardrails complement, but do not replace, careful prompt design.

Deliberately probing a system with adversarial inputs to find harmful, unsafe, or off-policy behavior before release. Findings feed back into prompts, guardrails, and evaluations.

A repeatable setup that runs a fixed dataset of inputs through the system and scores outputs against criteria. It enables regression detection when prompts or models change.

A curated set of representative inputs with expected outputs or rubric labels, used as the reference for measuring quality. It should cover common cases, edge cases, and known failure modes.

Using a model to score another model's output against a rubric. It scales subjective evaluation but needs calibration against human labels to avoid systematic bias.

Instrumenting a system to capture traces, token usage, latency, errors, and tool calls. It makes production failures diagnosable and supports cost and quality monitoring.

Reusing computed representations of unchanged prompt prefixes to cut cost and latency on repeated long contexts. It works best when static instructions precede variable content.

A cap on requests or tokens per unit time enforced by the provider. Production clients need backoff, retries, and queuing to handle throttling gracefully.

Re-sending failed requests after increasing delays, often with jitter, to recover from transient errors without overwhelming a struggling service. Non-retryable errors should fail fast.

Designing operations so repeating the same request produces the same result. It matters when retries or agent loops might execute a side effect more than once.

Delivering model output incrementally as it is generated, improving perceived responsiveness. It complicates validation and error handling because content arrives in pieces.

The time between a request and a usable response. It is shaped by model choice, prompt size, output length, and tool round-trips, and is often managed with streaming.

Sending each request to the cheapest or fastest model that meets its quality bar. It balances cost against capability, with harder tasks escalated to stronger models.

A predefined alternative path when a primary model call fails, such as a secondary model, cached answer, or graceful error message. It keeps user-facing features available during outages.