Support Retrieval: A Half-Reindexed Corpus and Tables That Fall Apart

Ranking.

1. Mixed embedding spaces. Articles are in the new model's space, PDFs in the old one. Distances between the two are meaningless, so PDFs - the pricing and compatibility content - are systematically under-retrieved. This alone can explain much of the numeric failure.
2. Table destruction. Fixed token windows cut rows from headers and units, so even when the right PDF is retrieved, the passage cannot answer the question.
3. No hybrid retrieval or reranking. Numeric and rare-token queries (part numbers, limits) are exactly where dense vectors underperform and lexical matching wins.
4. Chunk size vs. document shape. 512 tokens with no structure awareness is wrong for 1,900-token documents with section hierarchy.

Sequence. Reindex everything under one embedding model with an alias switch first - it is cheap and it unblocks measurement. Then fix table representation, then add hybrid retrieval and reranking, measuring recall@5 and numeric exact match after each step so the effects are attributable.

Why not prompt tuning first. The generator cannot cite a value that retrieval never returned. Tuning on a broken context teaches the team to optimize around a defect and contaminates the eval set.

Preserve structure at chunk boundaries. One chunk per table or per logical row group, with the header row repeated in every chunk and the section title and document version prepended as context. Convert to compact markdown or HTML rather than raw text so row/column relationships survive tokenization.

Keep units attached. Never emit a bare number. Write max payload: 512 MB (0.5 GB) and keep the condition that qualifies it (for the Standard tier). Units lost across chunks is a correctness bug, not a formatting issue - 512 MB and 512 GB are different answers.

Dual representation.

• A text form for semantic retrieval: a short natural-language summary of the table plus row-level statements such as Model X supports feature Y: yes, from firmware 4.2.
• A structured form (a table or key-value store) for exact lookup when the query is a single numeric fact. Route obvious numeric lookups to the structured path.

Matrices. Emit pairwise statements rather than a grid; a 20x30 grid retrieved as a grid cannot be reasoned over, but the relevant row statements can.

Chunking. Move to structure-aware chunking at section boundaries with heading paths, roughly 300–600 tokens, overlap only where prose continues across the boundary.

Retrieval path.

1. Metadata pre-filter on product, version, doc type, and region - cheap and removes whole classes of wrong answers.
2. Hybrid candidate generation: lexical (Oracle Text or BM25) plus vector, each returning top 50.
3. Fusion with reciprocal rank fusion, which needs no score calibration between the two systems.
4. Cross-encoder rerank the fused 50 down to 5, which is where most precision is recovered.
5. Generate with citations restricted to the reranked set, and refuse when the top rerank score is below a threshold.

Migration. Build a new index under a versioned alias, backfill all 71,400 documents with the current model, and verify counts and a sampled recall check before switching the alias. Keep the old index addressable for rollback. Never mix two embedding models in one index; store the model id and dimension in index metadata and reject writes that do not match.

Measurement. Track recall@5 and nDCG@10 overall and split by PDF vs. article - the split is what exposes the mixed-space bug. Add numeric exact match, citation precision, refusal correctness, and a per-section view (pricing, compatibility, limits). Re-run on every index rebuild and every model change, and log queries with zero results above threshold as a source-coverage backlog.