EGATSU-SP common mistakes
EGATSU-SP · Understand the traps and how to avoid them.
Treating "system prompt" and "user prompt" as interchangeable labels when asked to classify prompt components, so the candidate assigns the role instructions to the user turn or the task data to the system turn.
The terminology feels intuitive, so candidates answer from everyday usage rather than from the operational distinction: system-level content sets persistent behaviour and constraints, while user-level content carries the per-request task and data.
What to do instead
Before answering, restate the component's function in one line (persistent behaviour/constraint vs. per-request input) and map the label to that function. Practice by taking a sample prompt and sorting each line into system, user, and assistant roles with a one-sentence justification.
Confusing tokens with words or characters when estimating context-window fit, and answering that a 1,000-word document fits a stated token budget without conversion.
Candidates carry the informal "one token equals one word" shorthand and do not apply the actual subword behaviour, where a token is often a fraction of a word and non-English text or code consumes more tokens per character.
What to do instead
Memorize the working ranges: roughly 1.3 tokens per English word and roughly 1 token per 3-4 English characters, with higher ratios for code and CJK text. On any context-fit question, convert to tokens explicitly and show the arithmetic before choosing.
Writing requirements that specify the model or vendor ("use model X with temperature 0.2") instead of the observable behaviour the system must exhibit.
Candidates mirror implementation detail because it feels concrete and testable, missing that requirements must survive a change of model or provider and remain verifiable by an acceptance test.
What to do instead
Rewrite each requirement as a stimulus-response statement with a measurable threshold (for example, "given a 500-word support ticket, the system returns a category from the fixed list with at least 92% agreement against the labelled set"). Move model and parameter choices into design, not requirements.
Skipping the explicit statement of scope boundaries and out-of-scope inputs when analysing requirements, then failing questions about what the system should do with unsupported or adversarial requests.
Candidates focus on the happy-path capability list and treat refusal behaviour, input validation, and fallback handling as afterthoughts, so the requirement set is incomplete at the edges where most failures occur.
What to do instead
For every requirement, add the paired boundary question: what happens when input is empty, malformed, out of domain, or exceeds length limits? Record these as explicit requirements with defined fallback behaviour rather than leaving them implicit.
Choosing a single monolithic prompt or a single large model call for a task that clearly needs decomposition, retrieval, or tool use, because the design question mentions only one capability.
Candidates optimise for the simplest architecture and under-weight accuracy, latency, cost, and auditability trade-offs that the scenario implies but does not spell out.
What to do instead
Before selecting an architecture, list the constraints the scenario states (accuracy target, latency budget, data freshness, cost ceiling, audit trail) and check each candidate design against all of them. Justify the choice by naming which constraint eliminated the alternatives.
Assuming a larger context window removes the need for chunking or retrieval, and answering that the whole corpus can simply be pasted into the prompt.
The context limit is the most visible number, so candidates treat it as the only constraint and overlook attention dilution, cost per call, latency, and the fact that irrelevant context degrades answer quality.
What to do instead
Treat context capacity as one constraint among several. When a scenario involves a large or changing corpus, evaluate retrieval or chunking against cost, latency, and precision, and state the reason the full-context option was rejected.
Setting temperature, top-p, and max tokens without stating the objective they serve, and then being unable to explain why a deterministic-output requirement was configured with a high temperature.
Candidates memorize parameter names and default values but not the behavioural effect, so they map parameters to scenarios by association rather than by reasoning about output variability.
What to do instead
Anchor each parameter to an outcome: low temperature for classification and extraction where consistency matters, higher for divergent ideation, max tokens as a hard output-length guard rather than a quality control. For each configuration question, name the required output property first, then choose the parameter.
Evaluating only with a handful of hand-picked examples and reporting a single aggregate accuracy figure as evidence the system is ready.
Small convenience samples are fast and produce encouraging numbers, and candidates underestimate how much a curated set overstates performance on real, skewed, or adversarial input.
What to do instead
Design an evaluation set that states its size, its sampling method, and its coverage of edge cases and adversarial inputs. Report per-category results alongside the aggregate, and include a held-out set that was not used during prompt iteration.
Diagnosing a quality regression by immediately rewriting the prompt, without first isolating whether the cause is the model version, the retrieval layer, the input distribution, or the prompt itself.
Prompt editing is the most accessible lever, so candidates jump to it and mistake correlation for cause, wasting iterations and sometimes masking the real fault.
What to do instead
Follow a fixed triage order: confirm the change timeline (model, index, or code deploys), reproduce with a logged failing input, then vary one factor at a time. Only edit the prompt after the other layers are excluded, and keep the failing input as a regression case.
Treating prompt injection as solved by instructing the model to ignore malicious instructions, and answering that this instruction is a sufficient control.
The instruction reads like a security measure and candidates accept it at face value, missing that untrusted content in the context can override instructions and that defence must sit outside the model.
What to do instead
Describe layered controls: separate trusted instructions from untrusted retrieved content, constrain what tools and data the model can reach, validate and sanitise outputs before action, and require human confirmation for consequential operations. State explicitly that in-prompt instructions are not a security boundary.