Auditing an 88 Percent Land Cover Accuracy Claim

The headline pair hides more than it reveals.

• Rare classes are under-sampled. With 350 simple random points, dense forest (6%) receives about 21 points and water (4%) about 14. A user's accuracy of, say, 85% on 21 points has a confidence interval roughly 15 points wide - far too coarse to support any statement about dense forest, the class the department cares about most.
• Area is not estimated. Overall accuracy is a proportion of points, not of hectares. Estimating class area from a map requires the confusion matrix combined with mapped proportions, with a confidence interval on each area.
• Kappa adds little and can mislead. It is prevalence-sensitive; with agriculture at 52% it partly reflects class imbalance, and it answers no question the department actually asked.
• Per-class errors are absent. Producer's and user's accuracy per class are the numbers that determine whether the map can do a given job.
• Temporal mismatch. Field visits happen after the 2022 image date; a class change between image and visit is an error in the assessment, not in the map, unless interpreted against the image date.
• Non-response is unaddressed. Inaccessible points cannot simply be dropped; dropping them biases the estimate toward accessible, usually road-adjacent, classes.

Ask for: confusion matrix, per-class user's and producer's accuracy with intervals, and area estimates with intervals.

Design:

• Stratified random sampling by map class, which is the standard basis for both accuracy and area estimation with confidence intervals.
• Allocation. If a class must be reported individually, it needs a floor of about 50 points to give a usable interval. Seven classes at 50 already consumes the entire 350 budget with nothing left for non-response. So: either negotiate a larger budget, or formally drop rare classes from individual reporting and report them only as a grouped class. Do not silently under-sample and then quote a precise figure.
• Remaining points after floors go proportional to mapped area, with extra in the open-forest/scrub confusion zone, which is where most of the practical error will sit.
• Inaccessible areas. Pre-define a protocol: attempt access, then substitute the nearest accessible point within the same stratum and same landscape unit, and record every substitution. Report the proportion of substituted points by class. If a stratum is wholly inaccessible, exclude it from area estimates with an explicit caveat, or use high-resolution imagery interpretation as a documented fallback.
• Field protocol. Record the class by a defined rule set (minimum patch size, canopy threshold), geotag, photograph, and record the interpreter's confidence separately from the observed class.

Legitimate, with stated uncertainty:

• District-scale area estimates for the major classes (agriculture, scrub, open forest, built-up) with confidence intervals, used for planning and trend monitoring under a consistent, repeatable protocol.
• Prioritisation of where to send field staff, and identification of large contiguous patches.

Not legitimate:

• Enforcement notices against individual encroachment. A 30 m pixel cannot establish a parcel boundary or the date of occupation; the map is evidence of nothing at parcel scale, and legal exposure from false notices is serious. Enforcement must rest on ground survey and cadastral records.
• Sub-district area reporting for small administrative units, where a handful of pixels drives the number.
• Plantation verification, since plantations are typically under the minimum patch size and are confused with scrub and open forest; survival counts need field or very high resolution data.
• Precise claims about dense forest area. With the achievable sample, report it as a range; a change of two percentage points in a 6% class is well inside the noise.

Communicate all four points in a short cover note; most misuse of such maps comes from silence about limits, not from the map itself.