AWS GenAI Developer Pro cheat sheet

Amazon Bedrock Converse API provides a model-agnostic message format (system, user, assistant turns) so the same request code targets multiple foundation models.

InvokeModel accepts a raw model-specific JSON body; InvokeModelWithResponseStream returns chunks via an event stream for token-by-token streaming to clients.

Bedrock Knowledge Bases, Agents, and Guardrails are separate managed features; the Converse API does not by itself perform retrieval or tool execution.

Tool use (function calling) requires three steps: send tool definitions, receive a toolUse block, return a toolResult block, then re-invoke to obtain the final answer.

Bedrock provisioned throughput is purchased in model units and guarantees capacity; on-demand mode is billed per input/output token and subject to throttling.

Cross-Region Inference profiles route requests across regions to raise available throughput; the inference profile ID, not the bare model ID, is passed to the API.

Context window limits are counted in tokens across input plus generated output; exceeding the limit returns a validation error rather than silent truncation.

LangChain and LlamaIndex integrate with Bedrock through the Converse API; retries should use exponential backoff with jitter on ThrottlingException.

A Bedrock Knowledge Base pipeline is: data source sync, chunking, embedding, vector store write; query time does embedding, vector search, optional rerank, then prompt augmentation.

Chunking strategies include fixed-size with overlap, hierarchical, semantic, and no chunking; smaller chunks improve recall precision while larger chunks preserve context.

Vector stores supported include OpenSearch Serverless, Aurora PostgreSQL pgvector, Neptune Analytics, Pinecone, and Redis Enterprise Cloud.

Hybrid search combines keyword (BM25) and vector similarity; metadata filtering narrows candidates by attributes such as tenant, date, or document type before similarity ranking.

Retrieval quality is tuned by top-k, chunk size and overlap, embedding model choice, and reranking; low faithfulness usually indicates irrelevant retrieved passages.

The RetrieveAndGenerate API returns citations mapping generated spans to source chunks; Retrieve alone returns passages for custom prompt assembly.

S3 data sources require an IAM role with s3:GetObject and s3:ListBucket; the knowledge base sync job must succeed before new documents become retrievable.

Structured data sources (Redshift, Glue catalog) are queried with generated SQL rather than vector search, so schema descriptions improve accuracy.

Amazon Bedrock customization options: continued pre-training (unlabeled domain text), fine-tuning (labeled prompt-completion pairs), and distillation (teacher model outputs train a smaller student).

Fine-tuning in Bedrock produces a custom model with a dedicated model ID; it must be purchased as provisioned throughput to be invoked, not on-demand.

Training data for fine-tuning is JSONL with prompt/completion records stored in S3; a validation split measures generalization and detects overfitting.

Continued pre-training suits teaching domain vocabulary and style; fine-tuning suits teaching task behavior and output format; RAG suits injecting frequently changing facts.

Bedrock model evaluation supports automatic metrics (accuracy, robustness, toxicity) and human evaluation workflows with custom datasets.

LLM-as-a-judge evaluation uses a strong model with a rubric to score outputs; scores should be validated against a human-labeled sample.

Distillation reduces latency and cost by training a smaller model on high-quality outputs from a larger teacher model.

Continuous evaluation compares baseline versus candidate model on a fixed golden dataset before promotion; regression gates block deployment on metric drop.

Bedrock access is granted through IAM policies scoped to actions such as bedrock:InvokeModel and resource ARNs of specific models or inference profiles.

Knowledge base data source access uses a service role; the caller's identity and the knowledge base role are separate permission boundaries.

AWS KMS customer managed keys encrypt knowledge base vector stores, S3 data sources, and custom model artifacts; key policy must allow the service principal.

VPC endpoints (AWS PrivateLink) keep Bedrock and OpenSearch traffic off the public internet; interface endpoint policies can restrict which models are callable.

Amazon Bedrock does not use customer prompts or completions to train base models; data remains in the region and is encrypted in transit with TLS.

Prompt injection defenses include instruction hierarchy, input/output guardrails, least-privilege tool permissions, and allow-listing tool arguments.

Sensitive data detection uses Guardrails sensitive information filters or Macie classification on source buckets before ingestion.

Application-level tenant isolation is enforced with metadata filters on retrieval plus per-tenant IAM session tags and separate vector indexes where required.

CloudWatch metrics for Bedrock include Invocations, InvocationLatency, InputTokenCount, OutputTokenCount, and ThrottledRequests per model or inference profile.

Model invocation logging to CloudWatch Logs or S3 records request and response bodies; it must be explicitly enabled and requires IAM permissions.

Bedrock runtime errors to alarm on: ThrottlingException (429), ModelTimeoutException, AccessDeniedException, and ValidationException for oversized input.

Token cost is input tokens times input price plus output tokens times output price; output tokens are typically priced higher than input tokens.

Prompt caching and shorter prompts reduce cost; semantic caching of repeated queries avoids duplicate model calls.

Provisioned throughput lowers unit cost at sustained high volume; on-demand suits spiky or low traffic.

AWS X-Ray and OpenTelemetry tracing capture end-to-end latency across retrieval, model call, and guardrail evaluation segments.

Batch inference jobs process large offline workloads at lower cost than synchronous on-demand invocation.

Amazon Bedrock Guardrails apply content filters (hate, insults, sexual, violence), denied topics, word filters, PII filters, and contextual grounding checks.

Contextual grounding checks score grounding (answer supported by source) and relevance (answer addresses the question); below-threshold responses are blocked or flagged.

Guardrails can be applied to both user input and model output; ApplyGuardrail can evaluate text independently of a model invocation.

Automated reasoning checks validate model claims against formal policies and return which rules were violated.

Model cards and AWS AI Service Cards document intended use, limitations, and responsible AI considerations for transparency.

Governance controls include CloudTrail audit of model invocations, Bedrock model invocation logging retention, and tagging for cost and ownership.

Human-in-the-loop review is required for high-impact decisions; confidence thresholds and escalation paths route low-confidence outputs to reviewers.

Red-teaming with adversarial prompts tests jailbreak resistance; findings feed guardrail tuning and evaluation datasets.