INT8 Conversion Drops Small-Defect mAP on an Edge Device

The calibration set is the prime suspect. It was sampled randomly, so it is dominated by defect-free background, and the activation ranges recorded for INT8 scaling reflect mostly empty surface. Small-defect responses are low-energy and sparse; when the scale is set from background-dominated statistics, those activations are clipped or rounded away. That matches the pattern: overall mAP falls moderately, but small-defect recall collapses from 0.71 to 0.34.

Rebuild the calibration set deliberately:

• Use 300–500 images stratified by defect type, size bucket, lighting, and background, with a meaningful share of small defects.
• Include hard negatives that resemble defects, so scales are not set only by easy background.
• Run per-layer sensitivity analysis and keep the most sensitive layers in higher precision if the toolchain supports mixed precision.
• Re-evaluate per size bucket, not only aggregate mAP, because aggregate mAP hides the failure that matters.

A second check is preprocessing mismatch: if calibration images were resized or normalized differently from deployment, the recorded ranges are wrong regardless of content. Verify the exact AIPP or CPU preprocessing used at inference.

The current accounting is incomplete. The 28 ms covers model inference but not the 15 ms CPU resize and normalization, so the real path is 43 ms before camera transfer and post-processing. At a 33 ms budget, the pipeline already fails even though the model looks fast.

I would build the budget explicitly:

• Camera capture and transfer: measure, do not assume.
• Decode, resize, color conversion, normalization: move to device-side AIPP and hardware decode where available, so the CPU step shrinks toward zero.
• Inference: 28 ms measured.
• Post-processing (NMS, thresholding, coordinate rescale): usually CPU and often several milliseconds; measure it.
• Result transport and actuation: measure.

If AIPP removes most of the 15 ms, the path can fit near 30 ms. If not, INT8 has bought little end-to-end. The judgement is that optimizing inference alone is insufficient; the preprocessing location is an architectural decision, and moving it to the device is generally cheaper than accepting a slower model.

I would not choose between reverting to FP32 and deleting the failing evaluation cases. Both hide the problem.

Framework:

1. Quantify the gap by size bucket and by business cost. A recall loss on 8×8 defects may be acceptable if those defects are caught downstream; a loss on 14×14 defects may not be.
2. Test targeted fixes in cost order: mixed-precision layers for sensitive stages, higher input resolution only if the latency budget allows, tiling or ROI crops for small defects, and distillation from the FP32 teacher into a smaller student.
3. Measure each option in milliseconds and per-bucket recall, not aggregate mAP. A tiling approach doubles or quadruples inference cost and may break 33 ms; that trade must be explicit.
4. Consider a two-stage design: a fast INT8 detector proposes regions, and a lightweight classifier verifies small candidates. This often recovers small-defect recall at lower cost than running the full detector at high resolution.
5. If no option meets both constraints, renegotiate the requirement with numbers. For example, 20 FPS with 0.65 small-defect recall may be better than 30 FPS with 0.34. That is a business decision, but it must be presented with measured trade-offs, not by changing the test set.