The Chatbot With 91% Test Accuracy and a 46% Fallback Rate
A school in Guwahati launches a WhatsApp-style helpdesk bot for 1,100 students and parents. Nine intents cover fees, scholarship status, transfer certificate, exam routine, holiday list, bus routes, contact details, admission and result publication. The team hand-labels 1,100 queries, trains TF-IDF with logistic regression, and reports 91% accuracy on a random 20% hold-out.
In a two-week pilot, 46% of real messages fall through to 'Sorry, I did not understand'. Parents write mixed-script messages such as 'fees kiman hobo?' and 'TC lgbole ki lage', and many send greetings, thanks, or questions about a new canteen. 'Holiday list' has 240 training examples while 'scholarship status' has 18, and the model regularly confuses 'transfer certificate' with 'fee receipt'. The pilot also produced 380 real messages that nobody has labelled yet.
The principal concludes that 'AI does not work for Assamese' and wants to replace the bot with a list of keyword rules.
1. Reconcile 91% test accuracy with a 46% fallback rate. What does each number actually measure, and which parts of the gap are model error versus measurement artefacts?
Read the worked answer
The two numbers measure different things on different distributions, so neither contradicts the other.
91% is accuracy on a random split of a hand-labelled pool. That pool was written in the same style, length and vocabulary as the training data, so the test set is not an independent sample of reality. It also hides class imbalance: with 240 'holiday list' examples against 18 'scholarship status', a model can score 91% while performing badly on the small intents. Macro-F1 would expose this.
46% is not the error rate. The fallback fires whenever the top predicted probability is below the threshold, so correct but low-confidence answers also fall through, and genuine out-of-scope messages (greetings, thanks, the canteen question) count as failures when they are simply not one of the nine intents.
The real model error is the remainder: unfamiliar code-mixed phrasing, unseen vocabulary, and the TC/fee-receipt confusion. So the gap is roughly one part distribution shift, one part imbalance, one part threshold and intent-design, and one part real error. Saying 'AI does not work in Assamese' attributes all of it to the language, which the evidence does not support.
2. Propose a concrete repair plan using the 380 unlabelled pilot messages. Justify each step with the specific symptom it fixes.
Read the worked answer
Label the 380 pilot messages first. That gives a genuine sample, not a re-split of the training pool, and it is the only way to know what real traffic looks like.
Add two intents: 'out_of_scope' (greetings, thanks, canteen) and 'human_handoff'. Right now those messages are forced into one of nine classes or dumped in fallback, which inflates the apparent failure rate and annoys users.
Rebalance. 'Scholarship status' has 18 examples; write 40-50 paraphrases, and consider class weights so the small intents are not drowned. Report macro-F1 and per-intent recall from now on, not accuracy.
Switch the vectoriser to character n-grams, TfidfVectorizer(analyzer='char_wb', ngram_range=(3, 5), min_df=2), and normalise case and whitespace. This absorbs 'fees', 'feees', 'FEES' and transliterated spellings without a hand-built dictionary, which word-level TF-IDF cannot.
Separate the TC and fee-receipt pair by adding the phrases that actually discriminate them, and inspect the misclassified examples rather than guessing.
Finally, calibrate the confidence threshold on the labelled pilot sample instead of keeping the default, and log the top fallback messages weekly so drift is visible.
3. The principal wants to abandon the model for keyword rules. Evaluate that option and state how you would measure whether either approach is working.
Read the worked answer
Keyword rules are not worthless, but they fail differently rather than better. They handle exact high-frequency phrases with certainty and full transparency, and a teacher can edit them without a data pipeline. On the other hand, they break silently on spelling variation and code-mixing, they cannot rank intents by confidence, and every new phrasing becomes a manual ticket. 'fees kiman hobo?' matches only if someone thought of it. For a bot receiving 380 unseen messages in two weeks, the rule list will be perpetually behind.
The defensible design is hybrid: keep the classifier, put a short rule-based fast path in front of it for the ten exact phrases that recur daily, and route low-confidence messages to the human_handoff intent with a promise of reply time.
Measure the outcome, not the model. Track task completion rate (did the parent get the answer or reach a person), fallback rate on labelled real traffic, time to human reply, per-intent recall, and the weekly count of new phrasings that no intent covers. If completion stays flat after two iterations, the principal's instinct is right and the bot should be scoped down to the three intents it reliably serves.