Building up user confidence for the spaceborne derived global and continental land cover products for the Mediterranean region: the case of Thessaly

Ioannis Manakos, Christina Karakizi, Giannis Gkinis, Konstantinos KarantzalosarXiv 2017 · arXiv

Abstract

Satellite-derived land cover products are essential for environmental monitoring, yet discrepancies between maps often hinder user confidence. This paper presents a novel validation framework that incorporates expert confidence levels into accuracy assessments. By applying this "weighted accuracy" approach to three major products—Globeland30, Copernicus High Resolution Layers (HRLs), and Corine Land Cover (CLC) 2012—in the Thessaly region of Greece, the study demonstrates that accounting for inherent landscape complexity provides a more reliable measure of map performance. Results show that all products achieve high reliability (>84% weighted accuracy), with HRLs performing best in urban areas and CLC 2012 showing the greatest consistency across classes.

Introduction

The proliferation of Global and Continental Land Cover (C/GLC) products, driven by improved satellite sensors and open-access data policies, has created a wealth of information for researchers and policymakers. However, these products often disagree due to differences in spatial resolution, classification schemes, and processing methodologies. This creates a "confidence gap" for end-users.

Standard accuracy assessments typically rely on binary (correct/incorrect) evaluations of validation points. This approach overlooks the reality that some terrain features are inherently difficult to classify, even for experts. This paper addresses this gap by developing a systematic inter-comparison and validation workflow that acknowledges uncertainty through a weighted accuracy metric.

Study Area and Datasets

The research focuses on Thessaly, a Mediterranean region in Greece characterized by a complex mix of agricultural land, forests, and urban centers.

Figure 1: Map of the study area, Thessaly, Greece.

The study evaluated three prominent datasets:

  1. Corine Land Cover (CLC) 2012: A 100m resolution product with a hierarchical classification scheme.
  2. GIO High Resolution Layers (HRLs): 20m resolution layers focusing on specific themes like Imperviousness and Forest Type.
  3. Globeland30: A 30m global land cover product.

Methodology: Weighted Accuracy Framework

The authors developed a validation workflow centered on expert annotation and weighted metrics. A key component was the use of stratified sampling to ensure that rare but important land types were represented.

Figure 2: Details of the CLC2012-based stratified sampling used for validation.

Two experts manually labeled 539 points using high-resolution Google Earth imagery. For each point, they assigned a Confidence Level (CL):

  • CL #1 (>75% certain): Clear features.
  • CL #2 (25–75% certain): Moderate ambiguity.
  • CL #3 (<25% certain): High vagueness or borderline cases.

The Weighted Overall Accuracy (WA) formula was then applied, where errors in high-confidence areas (CL #1) counted more heavily against the map than errors in low-confidence areas (CL #3). This ensures that maps are not unfairly penalized for "hard" cases where human interpretation is also uncertain.

Key Results and Discussion

The analysis revealed that while low-confidence samples (CL #3) accounted for only 5% of the total, they were the primary source of disagreement between products.

  • Overall Performance: All three products achieved a minimum weighted overall accuracy of 84%. When using CLC2012-based sampling, HRLs reached the highest WA at 90%, followed by CLC 2012 (89%) and Globeland30 (86%).
  • Class-Specific Findings: HRLs proved superior for "Artificial Surfaces" (urban areas), while CLC 2012 was the most consistent across all categories. Globeland30 performed well in agriculture but struggled with Mediterranean water bodies, showing a Producer’s Accuracy as low as 26% in some tests.
  • Metric Sensitivity: Weighted Accuracy was consistently 1–3% higher than standard Overall Accuracy. This indicates that standard metrics tend to underestimate map quality by over-penalizing complex, mixed terrains.

Conclusion

By incorporating expert confidence, the weighted accuracy framework provides a more objective and fair assessment of land cover products. While the study highlights the high reliability of current spaceborne products in the Mediterranean context, it also underscores the importance of using independent reference layers for validation. Future work should expand this approach to other ecological zones and explore the use of ultra-high-resolution drone data to further refine ground-truth validation.

Summary updated

Abstract

Across globe and space agencies nations recognize the importance of homogenized land cover information, prone to regular updates, both in the context of thematic and spatial resolutions. Recent sensor advances and the free distribution policy promote the utilization of spaceborne products in an unprecedented pace into an increasingly wider range of applications. Ensuring credibility to the users is a major enabler in this process. To this end this study contributes with a systematic accuracy performance measurement and continental/global land cover layers' inter-comparison moving towards confidence built up. Confidence levels during validation and a weighted overall accuracy assessment were applied. Google Earth imagery was employed to assess the accuracy of three land cover products, i.e., Globeland30, HRLs and CLC 2012, for the years 2010 and 2012. Reported rates indicate a minimum weighted overall accuracy of 84%. Specific classes' performance deviations from the general trend were noted and discussed on the basis of an unbiased sampling approach. By integrating confidence levels during the ground truth annotation, stratified sampling on the several Corine Level 3 subclasses and the weighted overall accuracy assessment, the different aspects of the considered land cover products can be highlighted more objectively.
BibTeX
@misc{manakos2017building,
  author = {Ioannis Manakos and Christina Karakizi and Giannis Gkinis and Konstantinos Karantzalos},
  title = {{Building up user confidence for the spaceborne derived global and continental land cover products for the Mediterranean region: the case of Thessaly}},
  year = {2017},
  eprint = {1702.07890},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1702.07890},
}
Figure