LLM Assistant for Research Summaries
Meridian Investors, a boutique with $6bn AUM, is piloting a generative AI assistant for 40 analysts. It summarises broker research and earnings-call transcripts and retrieves internal research notes from the firm's document store. The vendor uses a public cloud large language model, states that client data is not used for training, and offers a 30-day termination clause but no audit rights or model documentation. In a 50-summary pilot, three summaries invented a revenue figure and one misstated a debt covenant. Analysts report 60% faster coverage. Compliance is concerned about recordkeeping of investment recommendations, data leakage, and model risk. The vendor says the issues are known hallucination behaviour and offers a prompt update. The COO wants a go/no-go decision before the next quarter.
1. Recommend a go/no-go decision with conditions.
Read the worked answer
I would not approve an unrestricted go-live. I would approve a conditional, limited production pilot for summarisation only, with human review and no direct use in investment recommendations until the controls are proven. The hallucination rate in the pilot is material: three invented revenue figures and one covenant error in 50 summaries is not a cosmetic issue, because analysts may rely on a confident but false output.
Conditions should include mandatory citation to source passages, a reviewer sign-off before any summary enters research records, blocking retrieval of client-identifiable or restricted documents, and a vendor contract amendment covering audit rights, model-change notification and data handling. The prompt update alone is insufficient because hallucination is a model behaviour, not a prompt defect. The reasoning is that the productivity gain is real, but the firm must bound the use case, the data and the reliance before scaling.
2. Design the validation and ongoing monitoring for the tool.
Read the worked answer
Validation should use a test set drawn from the firm's own documents with known answers, covering revenue figures, covenants, dates and entities. Each output should be scored for factual accuracy, completeness and citation correctness, with a pre-agreed tolerance. The pilot already shows the need for adversarial cases such as restated accounts, complex covenant definitions and documents with tables.
Ongoing monitoring should sample outputs monthly, track hallucination and correction rates by document type and analyst, and log every override. Model or prompt changes should trigger revalidation. Analysts should receive training on the tool's limitations and a clear escalation path for suspected errors. The reasoning is that model risk management requires evidence that the control works in production, not just a one-off vendor demonstration. Monitoring should also cover drift after vendor model updates, which the firm may not see without notification rights.
3. Address the recordkeeping and vendor risk concerns.
Read the worked answer
If an analyst uses a summary to form or support an investment recommendation, the firm must retain the source and the reviewed summary in the research record, with a clear indication of what was machine-generated and who approved it. The tool should not become an unlogged side channel. Compliance should define whether prompts and outputs are business records, and the firm should be able to reconstruct a recommendation from retained evidence.
Vendor risk is significant: no audit rights, no model documentation and a 30-day termination clause are weak for a tool touching research data. The firm should seek audit rights, security certifications, data residency and subprocessor disclosure, model-change notification, and clear deletion obligations. A fallback plan is needed if the vendor changes model behaviour or terminates. The reasoning is that outsourcing does not transfer accountability; the firm remains responsible for the investment record and for the data it places in the tool.