Retrieval-Augmented Generation and Vector Search
Domain 3 - Retrieval-Augmented Generation and Vector Search
This domain tests the RAG pipeline end to end and the OCI stores that implement retrieval. You should be able to choose between OCI Generative AI Agents (managed RAG), Oracle Database 23ai AI Vector Search, and OCI Search with OpenSearch, and explain chunking, embeddings, top-k retrieval, reranking, and grounding.
Core rules
- Pipeline: ingest → chunk → embed → index → retrieve top-k → rerank → build prompt with retrieved context → generate. Citations depend on passing source text and metadata into the prompt.
- The same embedding model and dimension must be used for documents and queries; mixing models produces meaningless similarity.
- Oracle Database 23ai stores vectors in a VECTOR column with vector indexes and supports COSINE, DOT, EUCLIDEAN, MANHATTAN, and HAMMING distances. Cosine is typical for normalized text embeddings.
- Chunk size trades recall against precision; overlap preserves boundary context. Metadata filters enforce document-level access control before generation.
- Hybrid search (keyword plus vector) helps with exact identifiers, SKUs, and names that embeddings blur.
Worked example
A corpus of 50,000 policy PDFs must answer only from documents a user can see. Reasoning: chunk at roughly 500–800 tokens with overlap; embed with Cohere Embed; store chunks plus an access-group metadata field in Oracle 23ai. At query time, filter by the user's group, retrieve the top 50 by cosine distance, rerank to the top 5, then prompt Command R+ with those passages and require citations. Reranking is needed because vector similarity alone often ranks plausible-but-irrelevant chunks highly.
Traps
Embedding queries with a different model; skipping reranking and blaming the LLM; chunks so large that retrieval returns whole documents; assuming RAG updates itself when source documents change - reindexing is required.