Handwritten Bengali account-opening forms: a 76% average that hides the dangerous field

Why the average misleads. It pools fields with completely different error costs. A wrong address is annoying and correctable when a letter bounces. A wrong amount is money posted to the wrong account, in a system with no undo. Averaging 58% and 92% treats those as equivalent, and it lets a strong field subsidise a weak one. The average also hides which errors are catastrophic: 3.1% of amount fields contain a numeral confusion between ০ and ৬, which for a figure like ৫০৬ versus ৫০০ changes the deposit by six rupees, but in a longer number changes it by a place value.

Metrics to report instead.

• Per-field exact match, never averaged.
• Character error rate for name, guardian name and address, because 1 character wrong in a 40-character address is not the same failure as a wholly wrong address.
• For amounts: exact match plus dangerous-error rate, defined as any digit substitution that changes the value. Report this specifically, not folded into accuracy.
• Field-level coverage: how often the system abstains rather than guesses.
• End-to-end form accuracy: the share of forms where all six fields are correct without human correction, since that is what determines staff time saved.
• Escape rate: errors that survive human confirmation. This needs a pilot with a second independent check of a sample.

Acceptance gates, before any posting is automated. Amount in figures and amount in words each at 99.5% exact on 500 unseen forms, and the two fields agreeing with each other in 99.9% of cases; date at 99%; name, guardian name and address may sit below that provided every one of those fields is manually confirmed. If the gates are not met, the system runs as pre-fill only, with no automation of amounts at all.

What the machine may do. Read the photographed form, pre-fill the six fields on screen, show the cropped image of each field beside its transcription, and assign a confidence score per field. It may not post anything.

What the human must do.

1. Confirm the amount twice, once in figures and once in words, with both on screen at the same time. The system parses the Bengali words into a number and compares it with the digits; if they disagree, the form is blocked from proceeding until a senior staff member resolves it against the image.
2. Confirm name and guardian name against the image, since these drive account linkage and duplicate detection.
3. Enter the address manually until it clears 90%, rather than accepting a 58% field, because a wrong address affects KYC and correspondence.
4. Sign off the form. The sign-off is the legal act of verification, and it belongs to a named person.

Safeguards.

• Confidence threshold: any field below threshold, or any amount field at all, is highlighted and cannot be skipped with a single keystroke.
• Keep the photographed form as the auditor's legible source document, stored against the account number with an image hash, retained per the society's record-retention rule.
• Reconcile daily: total of amounts posted versus total of amounts on the day's paper forms, with any difference investigated the same day.
• No bulk auto-posting, and no undo-free operation on the first month. Run a 100-form pilot with 100% second-person verification and measure the escape rate before scaling.
• Log for every form: model version, per-field confidence, every human edit, time taken, and which staff member confirmed. This log serves the auditor and is the retraining dataset.
• Keep a manual fallback (typed entry) that works if the pipeline is down, so branch service does not depend on the model.

Diagnosis. Addresses are the longest, most variable fields: house number, lane, village, post office, PIN. Handwriting varies between writers and between the same writer on different days. The archive of 900 forms is small for a long-tail vocabulary of village names, and the distribution is skewed: a few villages appear hundreds of times, many appear once or twice. Exact match is also the harshest measure for a 40-character field: one wrong character fails the whole field, so 58% may understate how close the model is. Finally, addresses are often written across lines or squeezed outside the printed box, so the box detector may be cutting text.

Data work, in order of expected payoff.

1. Normalise before scoring: expand or standardise common abbreviations, strip punctuation, and score per address segment (PIN, village, post office, house) so the team can see whether the failure is the PIN or the lane name.
2. Constrain decoding with a gazetteer: a list of valid PIN codes and village/post-office names for the society's catchment. A recognised PIN narrows the district, and village names can then be matched against the list rather than decoded character by character. This typically converts many near-misses into exact matches.
3. Increase labelled variety: sample forms stratified by writer, by form template version, and by village, rather than taking the first 900 forms in the archive. Target a few hundred examples of the previously rare addresses.
4. Fix capture: train and test on the same kind of photograph the branch will actually take, with a simple jig or guide so the form is flat and evenly lit, and check box detection separately from recognition.

Modelling. Fine-tune with address-specific augmentation (slight rotation, blur, ink variation), and consider a character-level model with a language-model correction over village names.

Decision rule. Ship address pre-fill only when segment-level accuracy shows the PIN and village at high confidence; otherwise keep the field manual. Staff time saved on amounts and dates already justifies the pilot.