AWS GenAI Developer Pro common mistakes
AWS GenAI Developer Pro · Understand the traps and how to avoid them.
Treating Amazon Bedrock model invocation as a single synchronous call and ignoring the Converse/InvokeModel distinction between streaming and non-streaming responses, so the candidate picks streaming for batch jobs or non-streaming for chat UIs.
Candidates memorize that Bedrock 'calls foundation models' but never internalize that Converse normalizes message formats across providers while InvokeModel is provider-native, and that streaming changes timeout and token-accounting behavior. Scenario stems about latency and partial output then feel interchangeable.
What to do instead
Build a decision table: interactive chat needs streaming for perceived latency; batch or Lambda-backed jobs need non-streaming with bounded timeouts. Practice mapping each scenario keyword (typing effect, long-running generation, token-by-token) to the correct API and response handling pattern.
Choosing a single large model for every task in a multi-step application instead of routing simpler subtasks to smaller or cheaper models, and failing to justify the routing logic when a scenario mentions mixed workloads.
Candidates default to 'most capable model wins' because exam prep emphasizes quality. They miss that cost, latency, and accuracy tradeoffs are usually the actual test point, and that model routing or a model cascade is the expected architecture.
What to do instead
For any scenario describing heterogeneous tasks (classification, extraction, summarization, reasoning), explicitly separate them and assign a right-sized model per task. Memorize that classification and extraction tolerate small models while complex multi-hop reasoning needs larger ones, and state the routing rationale in your reasoning.
Answering RAG questions by picking 'increase chunk size' or 'add more documents' without diagnosing whether the failure is retrieval (wrong chunks returned) or generation (right chunks, wrong answer).
RAG is taught as a pipeline diagram, so candidates pattern-match symptoms to the most familiar knob. They skip the diagnostic step of checking whether the retrieved context actually contained the answer, which determines whether to fix embeddings, chunking, or the prompt.
What to do instead
Always classify the failure first: if the answer was absent from retrieved context, fix retrieval (chunking strategy, hybrid search, metadata filters, reranking); if present but ignored, fix the prompt or model. Practice writing this two-branch diagnosis before selecting any option.
Confusing Knowledge Bases for Amazon Bedrock managed chunking and vector store options with self-managed OpenSearch or Aurora pgvector, and selecting a managed option when the scenario requires control over embedding model or index tuning.
The managed Knowledge Bases service hides chunking and embedding choices, so candidates assume it is always the 'AWS-recommended' answer. Scenarios that demand custom embeddings, hybrid search tuning, or existing vector infrastructure are testing exactly the opposite instinct.
What to do instead
Learn the boundary: managed Knowledge Bases for speed and low ops; self-managed vector store when you need custom embeddings, fine-grained index control, or reuse of existing search infrastructure. Map each scenario constraint (existing OpenSearch cluster, custom embedding model, minimal ops) to the correct side.
Recommending fine-tuning when the real problem is missing or stale knowledge, or recommending RAG when the problem is output style, format, or tone that the base model cannot follow.
Both techniques are presented as 'customization', so candidates treat them as interchangeable upgrades. The exam distinguishes knowledge injection (RAG) from behavior and format shaping (fine-tuning), and scenario wording about 'our documents' versus 'our house style' is the tell.
What to do instead
Apply the split rule: facts and frequently changing knowledge go to RAG; consistent format, tone, classification behavior, or task specialization go to fine-tuning. When a scenario mentions both, expect a combined answer and be ready to justify each component.
Treating a one-time evaluation as sufficient and failing to design continuous evaluation with automated metrics, human review, and regression gates after model or prompt changes.
Candidates study evaluation as a checkpoint before deployment rather than a lifecycle control. Scenarios about model upgrades, prompt iteration, or drift are testing whether you add ongoing evaluation and golden datasets, not whether you ran an eval once.
What to do instead
Memorize the continuous evaluation loop: curated golden dataset, automated metrics (accuracy, groundedness, toxicity), periodic human review sampling, and a regression gate in CI before any model or prompt change reaches production. Attach this loop to every 'after we update' scenario.
Relying on IAM alone for generative AI security and forgetting that Bedrock guardrails, prompt-injection defenses, and data-boundary controls are separate layers that scenarios often require together.
Candidates map 'security' to IAM roles and policies because that is familiar AWS content, then miss that model-level protections (guardrails, PII filtering, denied topics) and application-level protections (input sanitization, output validation) are distinct controls the exam tests by scenario.
What to do instead
Build a layered checklist: identity and least privilege (IAM, scoped model access), network isolation (VPC endpoints, PrivateLink), data protection (KMS encryption, no sensitive data in prompts), and content controls (guardrails, PII redaction). For each scenario, name which layers apply rather than stopping at IAM.
Ignoring data residency and cross-region inference implications, assuming a Bedrock call keeps data in the region where the application runs.
Cross-region inference profiles and multi-region model availability are newer, easily overlooked features. Candidates answer residency questions with generic encryption answers instead of addressing where inference actually executes.
What to do instead
Learn that cross-region inference routes requests across regions for capacity and that data residency requirements may forbid it. When a scenario cites regulatory residency, prefer in-region inference and explicitly rule out cross-region profiles; pair this with KMS and VPC endpoint controls.
Monitoring only infrastructure metrics (latency, errors, throttling) and omitting model-quality and cost signals such as token usage per request, groundedness drift, and per-tenant cost attribution.
Operational habits from traditional workloads carry over, so candidates watch CloudWatch error rates but not token consumption or output quality. Cost and quality regressions in generative AI are invisible to standard dashboards.
What to do instead
Design a dashboard with three tiers: infrastructure (latency, throttles, 5xx), model quality (groundedness, refusal rate, hallucination sampling), and cost (input/output tokens, cost per conversation, per-tenant tagging). Practice naming the metric that would catch each described symptom.
Treating responsible AI as a single content filter and missing the broader governance set: bias evaluation, transparency and disclosure, human oversight for high-stakes actions, and audit trails.
Guardrails are the most concrete, demo-friendly control, so candidates reduce 'responsible AI' to 'turn on guardrails'. Scenarios about hiring, lending, or medical triage are testing human-in-the-loop and evaluation requirements that guardrails alone do not satisfy.
What to do instead
Memorize a governance checklist: pre-deployment bias and safety evaluation, guardrails for content and PII, human review for consequential decisions, disclosure that users are interacting with AI, and logging for audit. For each high-stakes scenario, name the oversight control in addition to the filter.