OCI GenAI Pro common mistakes
OCI GenAI Pro · Understand the traps and how to avoid them.
Treating OCI Generative AI as a single monolithic endpoint and assuming every model supports chat, embeddings, and fine-tuning identically, instead of checking per-model capabilities before designing a solution.
Candidates study the service landing page and the playground demo, where a chat model is preloaded, then generalize. OCI Generative AI exposes distinct model families (cohere Command for chat/generation, cohere Embed for embeddings, Meta Llama and Mistral for chat), and fine-tuning support differs by model and by region.
What to do instead
Memorize the capability matrix: which model families serve chat vs. embeddings vs. fine-tuning, and that availability is region-scoped. When a scenario asks for embeddings, immediately rule out generation-only models; when it asks for custom weights, verify the model supports dedicated AI cluster fine-tuning.
Confusing on-demand inference with a dedicated AI cluster, and proposing on-demand hosting for a workload that requires fine-tuned custom models or guaranteed capacity.
Both appear under the same service console, and the word "dedicated" sounds like a billing tier rather than an architectural requirement. Candidates miss that custom/fine-tuned models must be hosted on a dedicated AI cluster with an endpoint, while base models can be called on-demand.
What to do instead
Anchor on the rule: base models via on-demand or dedicated; fine-tuned models require a dedicated AI cluster and a dedicated endpoint. If a scenario mentions fine-tuning plus production traffic, the answer involves cluster sizing (compute units) and endpoint creation, not an on-demand call.
Designing a RAG pipeline that embeds documents with one embedding model and queries with another, or that changes the embedding model after ingestion without re-indexing.
Embedding model choice feels like an implementation detail, and vector similarity still returns results, just semantically wrong ones. Candidates do not realize vectors from different models occupy incompatible spaces.
What to do instead
State explicitly that the same embedding model and version must be used at ingest and query time, and that switching models requires re-embedding the entire corpus. In scenario questions, treat an embedding-model change as a full re-index event, not a config toggle.
Answering RAG questions by describing prompt engineering alone ("instruct the model to only use the context") without addressing retrieval quality, chunking, and grounding verification.
Prompting is the most familiar lever, so candidates reach for it first. Exam scenarios usually describe stale or irrelevant answers, which are retrieval failures, not prompt failures.
What to do instead
Diagnose in order: is the right chunk being retrieved (chunk size, overlap, metadata filters, top-k), is the context being passed correctly, then is the prompt constraining the answer. Map each symptom to its layer and pick the fix at the correct layer.
Assuming the model retains memory of previous API calls, or that conversation history is persisted server-side by OCI Generative AI.
Chat playground sessions feel stateful, and consumer chatbot intuition implies memory. The inference API is stateless; continuity comes only from resending prior turns.
What to do instead
Design stateless inference explicitly: the application stores and resends conversation history (or a summary of it) within the context window. Watch for context-window overflow as a follow-on failure and plan truncation or summarization.
Ignoring token limits and context-window math, then proposing architectures that stuff entire documents into a single prompt.
Token counts are invisible in prose scenarios, and candidates estimate by character count or page count. Long-context models encourage the belief that retrieval is unnecessary.
What to do instead
Convert deliberately: tokens are roughly proportional to words but vary by language and tokenizer, and both input and output consume the window. When a scenario involves large corpora, argue for chunking plus retrieval rather than full-document prompting, and reserve headroom for the response.
Treating temperature, top-p, and max tokens as interchangeable "creativity" knobs, or setting high temperature for extraction and classification tasks.
All three live in the same parameter block and tutorials use them loosely. Candidates do not connect determinism to task type.
What to do instead
Tie parameters to task: near-zero temperature for extraction, classification, and structured output; higher temperature only for brainstorming or creative generation. Know that top-p and temperature interact, and that max tokens truncates output rather than controlling style.
Assuming OCI Generative AI performs automatic PII redaction, content moderation, or prompt-injection defense by default, and omitting these from the architecture.
Security is often treated as inherited from the cloud provider. OCI secures the infrastructure and the API boundary, but application-level safeguards for prompts, retrieved content, and outputs remain the customer's responsibility.
What to do instead
Separate the shared-responsibility line explicitly: OCI handles network, tenancy, and encryption in transit/at rest; the customer handles input validation, PII handling, output filtering, and injection defenses. In architecture scenarios, name the control and where it sits.
Confusing IAM policy scope for Generative AI - granting broad tenancy-level access or forgetting that access to the model endpoint and to the underlying data (Object Storage, vector store) are separate policy statements.
Candidates memorize a single policy example and assume it covers the whole pipeline. A RAG solution touches several resources, each needing its own authorization.
What to do instead
Write policies compartmentalized by compartment and resource type: generative-ai-family for model/endpoint use, plus object-family and vector-store permissions for the data path. Check least privilege in every scenario that mentions cross-team or cross-compartment access.
Skipping the governance and observability layer - no mention of logging, usage/cost tracking, model version pinning, or evaluation - and presenting a working demo as a production architecture.
Study material emphasizes getting a response from the API, and demos succeed without these controls. Candidates answer the functional question and miss the operational one.
What to do instead
For any "production" scenario, add the operational quartet: request/response logging with PII scrubbing, token usage and cost monitoring, pinned model versions with a change/re-evaluation process, and a quality evaluation loop (including human review for high-stakes outputs). Name these proactively.