Abstract
Satellite-derived land cover products are essential for environmental monitoring, yet discrepancies between maps often hinder user confidence. This paper presents a novel validation framework that incorporates expert confidence levels into accuracy assessments. By applying this "weighted accuracy" approach to three major products—Globeland30, Copernicus High Resolution Layers (HRLs), and Corine Land Cover (CLC) 2012—in the Thessaly region of Greece, the study demonstrates that accounting for inherent landscape complexity provides a more reliable measure of map performance. Results show that all products achieve high reliability (>84% weighted accuracy), with HRLs performing best in urban areas and CLC 2012 showing the greatest consistency across classes.
Introduction
The proliferation of Global and Continental Land Cover (C/GLC) products, driven by improved satellite sensors and open-access data policies, has created a wealth of information for researchers and policymakers. However, these products often disagree due to differences in spatial resolution, classification schemes, and processing methodologies. This creates a "confidence gap" for end-users.
Standard accuracy assessments typically rely on binary (correct/incorrect) evaluations of validation points. This approach overlooks the reality that some terrain features are inherently difficult to classify, even for experts. This paper addresses this gap by developing a systematic inter-comparison and validation workflow that acknowledges uncertainty through a weighted accuracy metric.
Study Area and Datasets
The research focuses on Thessaly, a Mediterranean region in Greece characterized by a complex mix of agricultural land, forests, and urban centers.
The study evaluated three prominent datasets:
- Corine Land Cover (CLC) 2012: A 100m resolution product with a hierarchical classification scheme.
- GIO High Resolution Layers (HRLs): 20m resolution layers focusing on specific themes like Imperviousness and Forest Type.
- Globeland30: A 30m global land cover product.
Methodology: Weighted Accuracy Framework
The authors developed a validation workflow centered on expert annotation and weighted metrics. A key component was the use of stratified sampling to ensure that rare but important land types were represented.
Two experts manually labeled 539 points using high-resolution Google Earth imagery. For each point, they assigned a Confidence Level (CL):
- CL #1 (>75% certain): Clear features.
- CL #2 (25–75% certain): Moderate ambiguity.
- CL #3 (<25% certain): High vagueness or borderline cases.
The Weighted Overall Accuracy (WA) formula was then applied, where errors in high-confidence areas (CL #1) counted more heavily against the map than errors in low-confidence areas (CL #3). This ensures that maps are not unfairly penalized for "hard" cases where human interpretation is also uncertain.
Key Results and Discussion
The analysis revealed that while low-confidence samples (CL #3) accounted for only 5% of the total, they were the primary source of disagreement between products.
- Overall Performance: All three products achieved a minimum weighted overall accuracy of 84%. When using CLC2012-based sampling, HRLs reached the highest WA at 90%, followed by CLC 2012 (89%) and Globeland30 (86%).
- Class-Specific Findings: HRLs proved superior for "Artificial Surfaces" (urban areas), while CLC 2012 was the most consistent across all categories. Globeland30 performed well in agriculture but struggled with Mediterranean water bodies, showing a Producer’s Accuracy as low as 26% in some tests.
- Metric Sensitivity: Weighted Accuracy was consistently 1–3% higher than standard Overall Accuracy. This indicates that standard metrics tend to underestimate map quality by over-penalizing complex, mixed terrains.
Conclusion
By incorporating expert confidence, the weighted accuracy framework provides a more objective and fair assessment of land cover products. While the study highlights the high reliability of current spaceborne products in the Mediterranean context, it also underscores the importance of using independent reference layers for validation. Future work should expand this approach to other ecological zones and explore the use of ultra-high-resolution drone data to further refine ground-truth validation.
Summary updated