Sentinel-2 Land-Cover Map: Accuracy Assessment and Corrected Area Estimation
A regional authority is mapping built-up expansion over a 1,000 km² jurisdiction using Sentinel-2 at 10 m. Between 2019 and 2024 the classified map reports built-up 210 km², cropland 640 km² and water 150 km². An accuracy assessment used 500 stratified random points allocated in proportion to mapped class area: 180 in built-up, 220 in cropland and 100 in water. Reference interpretation gave: of the 180 built-up points, 122 were built-up, 40 cropland and 18 water; of the 220 cropland points, 12 were built-up, 190 cropland and 18 water; of the 100 water points, 82 were water and 18 cropland. The consultant reports overall accuracy 0.79 and kappa 0.68 and concludes the map is fit for annual reporting and that the built-up area figure of 210 km² can be quoted directly.
1. Compute the user's and producer's accuracies for each class and interpret the pattern.
Read the worked answer
Column totals (reference): built-up 122+12+0 = 134; cropland 40+190+18 = 248; water 18+18+82 = 118.
| Class | User's accuracy | Producer's accuracy |
|---|---|---|
| Built-up | 122/180 = 0.678 | 122/134 = 0.910 |
| Cropland | 190/220 = 0.864 | 190/248 = 0.766 |
| Water | 82/100 = 0.820 | 82/118 = 0.695 |
| Overall | 394/500 = 0.788 |
Interpretation. For built-up the gap between producer's (0.91) and user's (0.68) accuracy is large and diagnostic: the map finds nearly all real urban, but only two-thirds of what it labels urban is urban. The 58 non-built-up points mapped as built-up (40 cropland + 18 water) are overwhelmingly commission errors. Water is misclassified as built-up because shallow, turbid or sediment-laden water has a similar spectral response to bare ground and roofs at 10 m.
The practical consequence is directional: the urban area is over-mapped, and it is over-mapped in a way the overall accuracy figure conceals. Cropland and water, by contrast, show the opposite pattern - good user's accuracy, weaker producer's accuracy - meaning they are under-mapped, squeezed out by the expanding built-up class. Reporting a single overall accuracy hides exactly the asymmetry a planning department needs to know.
2. Compute the design-based estimate of built-up area and compare it with the 210 km² quoted from the map.
Read the worked answer
The map area is a pixel count, not an estimate of the truth. With stratified random sampling and proportional allocation, each stratum's weight equals its mapped area proportion: W = 0.21, 0.64, 0.15.
Estimated built-up proportion of the jurisdiction:
- From built-up stratum: 0.21 × (122/180) = 0.21 × 0.6778 = 0.14234
- From cropland stratum: 0.64 × (12/220) = 0.64 × 0.05455 = 0.03491
- From water stratum: 0.15 × (0/100) = 0
- Total = 0.17725
Estimated built-up area = 0.17725 × 1,000 km² = 177.3 km²
Difference: 210 − 177.3 = 32.7 km², an overstatement of about 15.6%.
Why it matters. Every point in the cropland stratum that was actually built-up counts, weighted by cropland's large area, and every built-up-stratum point that was actually cropland is removed. The correction uses the reference data to undo the classifier's commission bias. Quoting 210 km² would overstate annual urban expansion by roughly a sixth - enough to distort a land-use budget or an infrastructure forecast.
Also report a confidence interval: with 500 points the standard error of the built-up proportion is a few tenths of a percent, so the correction is well outside sampling noise and is a genuine bias, not scatter. For change between years, run the same estimator per epoch rather than differencing pixel counts, which compounds bias.
3. Assess the consultant's use of kappa and recommend what should replace it in the report.
Read the worked answer
Kappa adds nothing here and is arguably misleading.
Kappa compares observed agreement with the agreement expected from the marginal totals of the confusion matrix. Those marginals are themselves products of the classifier and of the sampling design, so the "chance" baseline is not a meaningful quantity. Kappa conflates two independent things that users need separated: how much of the map is right in quantity, and how much is right in location. A single number cannot serve both, and two very different maps can return the same kappa - one that gets the totals right but places every patch wrongly, and one that places patches correctly but misallocates area between classes.
Replace it with a quantity/allocation decomposition of the disagreement:
- Quantity disagreement - the mismatch in class totals (this is what drives the 15.6% area error).
- Allocation disagreement - correct totals, wrong places (transposition between classes).
Report also: the confusion matrix itself with user's and producer's accuracies per class, the corrected area estimates with confidence intervals, and a note on the 10 m minimum mapping unit - patches below roughly 0.5 ha are unresolvable and are systematically lost, which biases change detection toward large contiguous urban blocks and away from incremental encroachment.
Finally, state the sample design in the report: 500 points stratified proportionally gives good precision on the dominant cropland class but relatively few water points (100), so the water estimates are the weakest and should not be quoted to three significant figures.