99% on Train, 61% on Validation, and a 95% Claim to Farmers

Two distinct problems are tangled together.

Overfitting. Training accuracy 99.2% against validation 61%, with validation loss turning upward after epoch 12 while training loss keeps falling, is the textbook signature: the network has memorised the training images rather than learned features that generalise. The gap is not noise, it is the model fitting individual pixels and backgrounds.

Leakage in the split. Because the same leaf was photographed repeatedly from slightly different angles, a random 70/15/15 split puts near-duplicate images of the same leaf in both training and validation. The validation set is therefore not an independent test - it is a memory test. True field accuracy is likely lower than even 61%, and the 95% claim in the report has no basis at all: no number in the setup supports it.

The planned fix makes both problems worse. Training for 100 epochs gives the network 70 more epochs to memorise; raising the learning rate tenfold increases instability on a dataset this small and may prevent convergence entirely. Neither change addresses capacity, data quantity or split quality.

Split first, by group. Group images by leaf identity and, better, by field and village. Assign whole groups to train/validation/test so no leaf appears twice across the boundary. If possible hold out an entire village as the final test set - that approximates a new farmer's field. This alone will drop the validation score and tell the truth.

Then fix the data. Collect more images, especially for bacterial blight (90 images is not enough for a five-class problem from scratch). Use augmentation that matches field variation - rotation, brightness, contrast, blur, mild crops - and avoid augmentations that destroy diagnostic texture. Balance the rare class with class weights or targeted collection rather than blind oversampling of the same 90 leaves.

Then the model. Replace the from-scratch CNN with transfer learning from a pretrained backbone, freezing early layers and fine-tuning the last block; keep dropout, add early stopping on validation loss (it starts rising around epoch 12), and use a modest learning rate with a schedule rather than 0.01.

Metrics. Per-class recall, not accuracy - missing bacterial blight is worse than a false alarm. Track the confusion matrix, macro-F1, and calibration. Report the held-out-village result as the headline, with the number of images per class stated alongside.

What to claim. Not '95% accurate'. After a proper group split, the honest statement is something like: 'on leaves from villages not used in training, the model identified the correct disease class in X of Y cases; for bacterial blight, recall was Z%'. Per-class figures, the test-set size and the villages covered belong in the same sentence as the headline number. The app must also state what it was not trained on - other crops, other diseases, low-light photos - so a farmer knows when the answer is out of scope.

Safeguards in the app:

• A confidence threshold: below it, the app says 'not sure, send to an expert' rather than guessing. Never force a five-way answer.
• Human referral path to the block agriculture officer, with the photo attached, so a wrong prediction is caught before a pesticide is bought.
• No prescription output - the model classifies an image; dosage advice is a separate, regulated decision.
• Logging with consent so real field errors are collected, plus a monthly review of misclassifications, disaggregated by village and phone model.
• Version disclosure, so the cooperative knows which model produced an old recommendation.

A pilot on two new villages, with the app's advice checked by an agronomist for a full season, is the minimum before any wider rollout.