TL;DR
This paper presents the first statistically rigorous, country-wide accuracy assessment of the GlobeLand30 2010 land cover dataset for China. Using a novel two-level stratified sampling approach and visual interpretation of Google Earth imagery, they found an overall accuracy of 84.2%, indicating GlobeLand30's high utility for many applications, while also highlighting areas for improvement in mapping complex landscapes.
What Problem Does This Paper Address?
Accurate land cover information is crucial for understanding Earth's natural resources and environmental processes. While global land cover products like GlobeLand30 offer valuable 30m resolution data, their reliability needs thorough validation, especially at a national scale. Previous assessments for China were either not statistically robust, used coarser resolutions, or lacked comprehensive coverage. This paper aims to fill this gap by providing a statistically sound, geographically stratified accuracy assessment of GlobeLand30 2010 over China, enabling users to better understand its strengths and limitations.
Methods
The authors employed a rigorous two-level stratified random sampling design. The first level stratified China into ten geographical regions, and the second level stratified by GlobeLand30's land cover classes. A total of 8400 validation sample pixels were collected. Reference data was acquired through visual interpretation of high-resolution Google Earth imagery by experienced interpreters. To account for potential ambiguities, a "less strict" agreement definition (matching map label with either primary or alternate reference label) was used for the main analysis, yielding an overall accuracy of 84.2%. A "stricter" definition (matching map label only with the primary reference label) was also used for comparison, resulting in a lower overall accuracy of 81.0%. Error matrices were constructed for each region and then aggregated using appropriate weighting to derive national accuracy estimates.
Datasets
- GlobeLand30 2010: The primary land cover dataset being assessed.
- Google Earth High-Resolution Satellite Imagery: Used as the primary source for visual interpretation to create reference land cover labels for validation pixels.
- Other available datasets (Bing maps, Yahoo maps, color composites of Landsat images): Utilized when Google Earth historical imagery was not of sufficient quality.
The paper does not introduce new benchmark datasets but rather establishes a robust validation methodology for an existing one.
Key Insights and Ideas
- The importance of stratified sampling: The two-level stratification (geographic and thematic) is crucial for obtaining statistically sound and regionally representative accuracy estimates, allowing for both national and regional reporting.
- Defining "agreement" matters: The choice of agreement definition (primary vs. primary/alternate reference label) significantly impacts accuracy estimates. The "less strict" definition, which accounts for potential ambiguities in visual interpretation, provides a more realistic assessment of the product's utility for many applications.
- Spatial heterogeneity drives accuracy variations: Accuracy is not uniform across China. Regions with more complex and fragmented landscapes, or where certain land cover classes are rare, tend to exhibit lower accuracies.
- Class-specific challenges: Certain land cover classes, like shrubland, wetland, and artificial surfaces, are more prone to misclassification, highlighting the need for improved classification algorithms or protocols for these categories.
Key Results
- National Overall Accuracy: 84.2% (using the less strict agreement definition).
- National Overall Accuracy (stricter definition): 81.0%.
- High accuracies for dominant classes: User's and producer's accuracies were generally above 80% for cultivated land, forest, permanent snow and ice, and bareland.
- Specific class accuracies: User's accuracy for water bodies was high at 94.2%, while wetland and artificial surface accuracies were lower (82.1% and 79.8%, respectively).
- Regional variations: Overall accuracies ranged from 76.0% to 90.3%, with northern regions generally showing higher accuracies than southern regions.
- Misclassification patterns: Dominant classes were sometimes overestimated, and spectrally similar classes (grassland, shrubland, forest) were difficult to distinguish. Fragmented patches and "salt-and-pepper" noise were also identified as sources of error.
Limitations and Future Work
- Limitations:
- The "less strict" agreement definition, while practical, might mask some finer classification errors.
- The study focused on 2010 data; land cover is dynamic, and accuracy may change over time.
- The visual interpretation process, while rigorous, is still subject to interpreter bias and the quality of available imagery.
- The paper notes that areas with heterogeneous landscapes and scattered small patches require improved mapping methods.
- Future Work:
- The authors suggest that future global land cover mapping at fine spatial resolution needs to address challenges in mapping scattered classes of small extents and spectrally mixed pixels.
- Further research is needed to develop and refine mapping protocols to better distinguish classes prone to misclassification, such as grassland, shrubland, and artificial surfaces.
- The GlobeLand30 production and validation teams can benefit from these findings to improve future iterations of the dataset.
Summary updated