Proseware Legal's Document Assistant

Retrieval-augmented generation, decisively, and on three independent grounds:

1. The insurer constraint. Fine-tuning is training. Training on client contracts would breach the 'no client text used to train any model' requirement outright - the proposal is disqualified before its technical merits are discussed.
2. The weekly-changing manual. Fine-tuning bakes knowledge into weights; every policy change means a retrain, an evaluation cycle and a redeploy. With RAG you re-index the changed documents and answers reflect the change on the next query. A weekly cadence makes fine-tuning operationally unworkable.
3. Citation and auditability. RAG retrieves passages that already have document IDs and page numbers, so a citation is a reference to a retrieved chunk, not a claim generated from memory. Fine-tuning gives you no handle on where an answer came from, which is exactly the audit trail the insurer and the partners are asking for.

What fine-tuning is legitimately for here: response style, tone and format conventions - but that is a later optimisation, not the knowledge mechanism. Build RAG on Azure AI Search with an embedding index, chunk documents by clause or section, and keep document-level access control in the index.

What retrieval gives you. The prompt contains actual passages with identifiers, and the instruction to answer only from those passages. That converts 'recall a case name' into 'quote from the supplied text', which is a far easier task and where hallucination rates drop sharply. Because the citation points at a chunk the user can open, an error becomes checkable rather than invisible.

Why it is not sufficient. The model can still: mis-attribute a passage to the wrong document when several are retrieved; paraphrase a clause so the citation is technically present but supports the wrong proposition; or answer fluently when retrieval returned nothing relevant. Retrieval quality is also a failure mode - if the right clause was never retrieved because chunking cut it in half, the model has nothing to be faithful to.

So add: (a) an abstain path - if the top retrieved relevance score is below a threshold, return 'not found in the corpus' rather than an answer; (b) per-document security trimming so a user cannot be shown a clause from a matter they cannot open - which also addresses the blended-client incident; (c) a groundedness evaluation set of question/answer/source triples run on every index or prompt change; (d) citation rendering that links to the exact passage, meeting the one-click verification requirement.

No training on client text. Use Azure OpenAI with the data, privacy and security commitment that prompts and completions are not used to train or improve the foundation models, and do not enable any fine-tuning on client content. Keep the corpus in your own search index, which is storage you control, not model weights. Put this in writing for the insurer - the commitment is contractual as much as technical.

Auditability. Log, per request: user identity, timestamp, the query, the retrieved chunk IDs with document and page references, the model deployment and version, and the final answer. Retain logs under the firm's matter-retention rules. Evaluations of groundedness and citation accuracy should be versioned alongside the prompt and index configuration, so a behaviour change can be traced to a change.

Residual risks to state rather than hide. Logs of queries themselves contain client information and are a new store to protect. The relevance threshold will produce some false 'not found' answers that erode trust. Content filtering may block legitimate legal vocabulary. And a correct-looking answer with a correctly retrieved but inapplicable clause is still possible - which is why the partner's one-click verification is the control of last resort, not a convenience.