AAIA common mistakes
AAIA · Understand the traps and how to avoid them.
Treating supervised, unsupervised, and reinforcement learning as interchangeable labels and picking whichever term sounds closest to the scenario, without first asking whether labelled target data exists.
Candidates memorise definitions in isolation and never build the decision rule: labels present means supervised, structure discovery means unsupervised, reward-driven sequential decisions means reinforcement learning.
What to do instead
Before reading the answer options, write one line about the data: 'labels exist / no labels / reward signal over time'. Then eliminate any option from the wrong family, even if its wording matches the scenario vocabulary.
Confusing classification with regression when the output is numeric, and confusing precision/recall/F1 with accuracy when the classes are imbalanced.
Numeric-looking outputs (scores, probabilities, risk levels) invite 'regression' answers, and accuracy is the most familiar metric, so it feels safe.
What to do instead
Ask what the output physically is: a discrete category means classification, a continuous quantity means regression. For metrics, ask which error is costlier (false positive versus false negative) and choose the metric that penalises it; reject accuracy whenever one class dominates.
Describing how a foundation model works as 'it searches the internet and copies the best answer', or attributing factual recall to a retrieval database that was never mentioned.
Everyday chatbot experience creates a mental model of lookup, and vendor marketing blurs the line between pretraining, fine-tuning, and retrieval augmentation.
What to do instead
Anchor on the pipeline: tokenisation, learned embeddings, next-token prediction over parameters, then optional adaptation (fine-tuning, RAG, prompt engineering). When a question mentions no external store, the knowledge is in the weights; when it mentions a document store queried at inference time, that is RAG.
Mixing up fine-tuning, RAG, prompt engineering, and continued pretraining when asked for the cheapest or most appropriate way to change model behaviour.
All four 'customise' a model, and the exam scenarios often describe a business goal rather than a technical constraint, so candidates pattern-match on keywords like 'knowledge' or 'style'.
What to do instead
Map the need to the technique: instruction/format changes without new facts means prompt engineering; new or frequently changing private facts means RAG; consistent style, tone, or task behaviour means fine-tuning; broad domain adaptation on large unlabelled corpora means continued pretraining. Also check cost and latency constraints, since RAG adds retrieval latency and fine-tuning adds training cost.
Assuming a foundation model's stated context window is fully usable, or that a larger context window removes the need for chunking and retrieval.
Model cards advertise maximum context length, and candidates extrapolate that any document fits and is used equally well.
What to do instead
Treat context length as a hard ceiling that also consumes output tokens and system prompts, and remember that recall degrades over very long inputs. For large corpora, prefer chunking plus retrieval and cite the passages used, rather than stuffing everything into one prompt.
Choosing a foundation model for a task purely on benchmark leaderboard position, ignoring latency, cost per token, modality support, context limits, and data-residency constraints stated in the scenario.
Benchmarks are the most visible comparison and candidates treat them as the only selection criterion.
What to do instead
Build a short requirement checklist from the scenario (modality, latency budget, cost ceiling, region, context size, customisation needs) and eliminate models that fail any hard constraint before comparing quality. Quality then breaks ties among survivors.
Recommending a single large general model for every workload instead of matching model size and architecture to the task, such as using a large text model for image classification or a huge model for simple extraction.
Candidates equate 'most capable' with 'best choice' and overlook that small models or task-specific models can meet accuracy needs at far lower cost and latency.
What to do instead
State the task type first (text generation, summarisation, embedding, image classification, speech), then select the smallest model class that satisfies the accuracy requirement. Justify with the scenario's cost or latency signals.
Treating responsible-AI principles as a checklist to recite rather than applying them: for example, claiming a model is unbiased because the training data was large, or proposing explainability as a fix for a privacy problem.
The principles (fairness, explainability, privacy, safety, transparency, verifiability) are learned as definitions, so candidates match the word in the question to the word in the answer instead of to the actual harm.
What to do instead
For each scenario, name the specific harm and the affected stakeholder first, then select the control that addresses that harm: bias evaluation and mitigation for fairness, data minimisation and consent for privacy, human review and guardrails for safety, model cards and disclosure for transparency. Reject any answer that uses one principle to solve another's problem.
Confusing security controls with governance controls, such as answering a question about prompt injection with 'add a model card', or answering a question about audit trails with 'increase temperature'.
Security, compliance, and governance vocabulary overlaps, and candidates answer with the most familiar term rather than the control category the question targets.
What to do instead
Classify the requirement: preventing adversarial manipulation is security (input validation, prompt-injection defences, least-privilege tool access); proving who did what is governance and logging (immutable audit trails, lineage); meeting a regulation is compliance (data residency, retention, documented risk assessment). Choose a control from the matching category.
Assuming that because a foundation model is accessed through a managed API, the shared responsibility model makes the provider responsible for data protection, access control, and output handling.
Managed services feel 'handled', and candidates transfer the provider's responsibility for infrastructure to the customer's data and application layer.
What to do instead
Apply the split explicitly: the provider secures the service infrastructure and model hosting; the customer owns data classification, identity and access management, prompt and output handling, logging, and regulatory obligations for the data sent to the model. In any scenario question, assign the control to the party that owns that layer.