Model Selection, Prompt Engineering, and Context Management
Selection is a tradeoff, not a ranking
Model selection questions reward reasoning about task complexity, latency budget, cost per token, context window, and required capabilities such as tool use, structured output, or vision. Know that larger models generally improve reasoning and instruction-following but cost more and add latency; smaller models suit high-volume classification, extraction, and routing. The professional move is tiered routing: cheap model for easy traffic, escalate on low confidence or task type.
Prompt engineering that holds up
Understand the durable techniques: explicit role and task framing, few-shot examples chosen for coverage rather than volume, delimiters to separate instructions from data, chain-of-thought for multi-step reasoning, and output schemas for machine consumption. Prefer clear instructions over clever phrasing. Place long reference material before the question when it helps recall, and keep the most important instructions salient.
Context management
Context is a budget. Know strategies: summarization of older turns, sliding windows, retrieval instead of stuffing, and structured scratchpads for agent state. Watch for lost-in-the-middle effects and context rot from irrelevant tokens. Caching stable prefixes reduces cost and latency.
Traps
Options that say "use the largest model for everything" or "put every document in the prompt" are usually wrong. Beware prompts that over-specify formatting at the expense of correctness, and few-shot sets that leak the test distribution. A common trap: confusing a longer context window with better recall.
Rule of thumb
Choose the smallest model that passes your eval set for the task, and manage context by relevance, not by capacity.