watsonx GenAI Engineer: Credit-memo summarization: high-risk approval in six weeks

Credit-memo summarization: high-risk approval in six weeks

A bank built a credit-memo summarization assistant on Granite-13b for its commercial lending analysts. The assistant condenses credit files of up to 200 pages into a two-page summary covering borrower background, facility structure, covenant package, and key risks. Analysts remain the decision-makers, but the summary is the first thing they read. Internal model risk policy classifies this as a high-risk use case; the vendor model card gives no training data details. Offline evaluation on 300 memos shows 88% factuality when scored by an LLM judge and 74% when scored by human reviewers. In 6% of summaries, at least one covenant figure (threshold, margin, or maturity) is wrong. The governance team has asked for a model inventory entry, risk assessment, factsheet, evaluation evidence, drift and factuality monitoring, and a human-oversight design. The business wants production in six weeks; a full monitoring baseline is estimated at ten.

1. What must be in place before go-live, and what can be deferred with compensating controls?

Read the worked answer

Separate controls that prevent harm from controls that detect degradation.

Non-deferrable before go-live:

Risk classification and model inventory entry, because the use case is already designated high-risk and proceeding without the record is a policy breach.
Factsheet and model card documenting intended use, out-of-scope use, the vendor's missing training data disclosure as a known limitation, and the evaluation results.
Evaluation evidence on the harm that matters: covenant figure accuracy, not aggregate factuality. A 6% figure error rate on financial terms is the material risk.
Human oversight design: analysts must review highlighted covenant and financial figures against source pages before relying on the summary; no downstream automation may consume the summary directly.
Incident and escalation process, plus logging of inputs, outputs, and analyst overrides.

Deferrable with controls:

Full drift and factuality monitoring can launch in shadow mode alongside production, with a 30-day baseline and weekly human sampling in the interim. The compensating control is the mandatory analyst review of figures plus a sampled quality review of 50 summaries per week, reported to the model owner.

State the deferral explicitly in the risk assessment with an owner, a date, and the interim control - not as a verbal agreement.

2. The LLM judge says 88% factuality and humans say 74%. How do you reconcile them and set a threshold?

Read the worked answer

The human score is the ground truth; the judge is an instrument that must be calibrated. The 14-point gap is itself a finding - an uncalibrated judge will be used for continuous monitoring, so it must be fixed before it becomes the production metric.

Reconciliation steps:

Have humans label a stratified sample of 100 summaries, including the ones where the judge and humans disagreed, with a defined rubric: what counts as a factual error, and how errors are weighted by severity.
Measure judge agreement with humans (precision/recall on error detection). If the judge misses wrong covenant figures - likely, since it may not verify numbers against source - it cannot be the primary monitor for those.
Recalibrate the judge with explicit rubric and few-shot examples, or replace it for numeric claims with a deterministic check: extract each covenant figure from the summary and confirm it appears verbatim in the source page.

Threshold setting should be harm-based, not a single factuality number. Set covenant figure accuracy at or near 100% with source-span match required, because that is the failure that could mislead a credit decision. For narrative content, set an error-rate threshold from the human baseline (for example, no more than 1 material factual error per 10 summaries) and monitor it with the calibrated judge plus human sampling.

Report both metrics and the disagreement rate; regulators and internal audit will ask which one is real.

3. Design the human oversight and production monitoring so that the 6% figure error rate cannot cause harm.

Read the worked answer

Oversight design starts from the failure: wrong covenant thresholds, margins, and maturities. Make figures verifiable rather than trusting the prose.

Render every numeric claim with an inline citation to the source page and highlight it in a side-by-side viewer. The analyst confirms or corrects in one click.
Mark any figure the system could not ground in the source as unverified and block it from being copied into downstream documents.
Require an explicit analyst acknowledgment before the summary can be attached to the credit file, and log the acknowledgment, corrections, and time spent.
Do not allow the summary to feed automated covenant monitoring or rating tools; keep it advisory.

Monitoring:

Input drift: document types, page counts, lender templates, OCR quality. Alert on new templates and on documents outside the training distribution.
Output quality: weekly human review of a random sample plus all summaries where grounding failed; track material error rate, figure error rate, and override rate.
Override rate as a leading indicator: if analysts correct figures more often over time, quality is degrading even if the judge score is flat.
Calibration check: whenever the judge or prompt changes, re-run the human-labeled benchmark.

Route alerts to a named model owner with a runbook: pause the feature, fall back to non-generated extraction, and notify risk. Review the monitoring thresholds quarterly.