Multi-scale accuracy assessment of land cover datasets based on histo-variograms

Liujun Zhu, Pengfeng Xiao, Feng Xue-zhi, Wang Zuo, Jiang LuyuanNational 2013 · DOI

TL;DR

This paper introduces a novel multi-scale accuracy assessment method for land cover datasets using "histo-variograms." This approach allows for more comprehensive accuracy evaluation at both pixel and sub-pixel levels, addressing the limitations of traditional single-scale methods and providing richer information about dataset performance across different spatial resolutions.

What Problem Does This Paper Address?

Assessing the accuracy of large-scale land cover datasets is crucial for their reliable application in various fields like global change research and ecosystem modeling. However, existing methods often focus on a single spatial scale, failing to capture how dataset accuracy varies with different resolutions. This is problematic because land cover datasets are derived from satellite imagery with varying spatial resolutions, and their accuracy can differ significantly depending on the land cover type and the spatial context. Furthermore, issues like spatial misalignment between reference data and the datasets can introduce errors that single-scale assessments might overlook. This paper aims to bridge this gap by proposing a method that evaluates dataset accuracy across multiple scales.

Methods

The core of the proposed method lies in the "histo-variogram" and a concept called "standing-pixels."

  • Histo-variogram: This technique analyzes how the histogram of an image changes as its spatial resolution is aggregated (upscaled). By examining a series of histograms at different aggregation levels, it provides insights into the spatial characteristics of the data.
  • Standing-pixels: Traditionally, a "standing-pixel" is formed when all pixels within an aggregation window have the same land cover type. This paper relaxes this definition to "non-strictly defined standing-pixels." This means that even in mixed pixels, if the dominant land cover type exceeds a certain threshold, it's considered a standing-pixel for that dominant type. This allows for more robust sampling, especially for land cover datasets.

The method then applies these concepts in two ways:

  1. Pixel-scale accuracy assessment: Strictly defined standing-pixels are used as sample points to directly assess dataset accuracy. This method is less sensitive to spatial registration errors.
  2. Sub-pixel scale accuracy assessment: Non-strictly defined standing-pixels and their corresponding histo-variograms are used to evaluate the accuracy of the dominant land cover type within mixed pixels. This involves classifying standing-pixels into "structural" (based on spatial aggregation and fragmentation) and "non-structural" dominant types to understand how spatial characteristics influence accuracy.

Datasets

The study evaluated five widely used large-scale land cover datasets:

  • University of Maryland (UMD)
  • IGBP DISCover
  • MOD12Q1-2001
  • Global Land Cover 2000 (GLC2000)
  • GlobCover2009

The reference data used for the accuracy assessment consisted of three classified Landsat TM/ETM+ images from 1991, 2001, and 2009, covering a typical region in northern Zhejiang Province, China. These reference images were preprocessed and achieved an overall accuracy greater than 85% with kappa coefficients above 0.80.

Key Insights and Ideas

  • Multi-scale is crucial: The study strongly emphasizes that a single accuracy metric is insufficient. Understanding how accuracy changes with scale provides a much richer picture of a dataset's strengths and weaknesses.
  • Standing-pixels overcome registration issues: The use of standing-pixels, especially the relaxed definition, helps to mitigate errors caused by spatial misalignment between reference data and the datasets being evaluated.
  • Sub-pixel analysis reveals spatial relationships: The sub-pixel assessment effectively links the accuracy of a land cover type to its spatial characteristics (area and structure) within mixed pixels. This is vital for understanding how well datasets capture boundaries and heterogeneous areas.
  • Dominant type matters: The accuracy of mixed pixels is largely determined by the accuracy of the dominant land cover type within them.

Key Results

  • The proposed multi-scale accuracy assessment method provides a more comprehensive evaluation of land cover datasets, offering richer multi-scale accuracy information.
  • Pixel-scale accuracy assessment can reduce errors from spatial misalignment, leading to more objective results.
  • Sub-pixel scale assessment effectively reflects the relationship between accuracy and the spatial structure and area of dominant land cover types.
  • Generally, GLC2000 showed the highest overall accuracy, while UMD had the lowest.
  • Accuracy for forest and cropland varied significantly across datasets, with some datasets overestimating and others underestimating these classes. Water and urban classes generally showed lower accuracies.
  • For sub-pixel analysis, accuracy generally increased with the area of the dominant land cover type, especially for areas larger than 0.5 km².
  • The spatial structure (fragmentation and aggregation) of land cover types influenced accuracy, with higher fragmentation leading to more random accuracy assessments.

Limitations and Future Work

  • The study acknowledges that the number of non-strictly defined standing-pixels for non-structural dominant types (NSDT) was scarce, accounting for less than 5% of all standing-pixels. This limited their influence on the overall sub-pixel accuracy assessment.
  • The authors suggest that for regions with low spatial fragmentation, considering the spatial structure of sub-pixel dominant types might not be necessary.
  • The multi-scale accuracy assessment method can also be applied to evaluate low-resolution data using high-resolution reference data.

Summary updated

Abstract

针对目前精度评价尺度单一的问题,提出基于直方变差图的多尺度精度评价方法,åˆ†åˆ«åœ¨åƒå ƒå°ºåº¦å’Œäºšåƒå ƒå°ºåº¦è¿›è¡ŒåœŸåœ°è¦†ç›–æ•°æ®é›†ç²¾åº¦è¯„ä»·ã€‚åœ¨åƒå ƒå°ºåº¦åˆ©ç”¨é©»ç‚¹ä½œä¸ºé‡‡æ ·å·¥å ·ç›´æŽ¥è¯„ä»·æ•°æ®é›†ç²¾åº¦;äºšåƒå ƒå°ºåº¦ä¸Š,åˆ™åˆ©ç”¨éžä¸¥æ ¼å®šä¹‰çš„é©»ç‚¹å’Œé©»ç‚¹ç›´æ–¹å˜å·®å›¾å¯¹ä¸åŒé¢ç§¯å’Œç©ºé—´ç»“æž„çš„ä¼˜åŠ¿ç±»è¿›è¡Œç²¾åº¦è¯„ä»·ã€‚å¹¶ä»¥æµ™æ±ŸåŒ—éƒ¨å ¸åž‹åŒºåŸŸä¸ºå®žéªŒåŒº,Landsat TM/ETM+为参考数据,对UMD、IGBP DISCover、MOD12Q1-2001、GLC2000、GlobCover2009等5种大尺度土地覆盖数据集进行多尺度精度评价实验。结果表明,å¤šå°ºåº¦ç²¾åº¦è¯„ä»·æ–¹æ³•èƒ½å¤Ÿå ¨é¢åœ°è¯„ä»·åœŸåœ°è¦†ç›–æ•°æ®é›†çš„ç²¾åº¦,æä¾›æ›´åŠ ä¸°å¯Œçš„å¤šå°ºåº¦ç²¾åº¦ä¿¡æ¯ã€‚åƒå ƒå°ºåº¦ç²¾åº¦è¯„ä»·å¯åœ¨ä¸€å®šç¨‹åº¦ä¸Šæ¶ˆé™¤ç”±äºŽå‚è€ƒæ•°æ®ä¸Žæ•°æ®é›†é—´çš„ç©ºé—´åŒ¹é é€ æˆçš„è¯¯å·®,è¯„ä»·ç»“æžœæ›´åŠ å®¢è§‚;äºšåƒå ƒå°ºåº¦ç²¾åº¦è¯„ä»·èƒ½æœ‰æ•ˆåæ˜ äºšåƒå ƒå°ºåº¦ä¼˜åŠ¿åœ°ç‰©é¢ç§¯åŠç©ºé—´ç»“æž„ä¸Žç²¾åº¦çš„å ³ç³»ã€‚
BibTeX
@inproceedings{zhu2013multi,
  author = {Liujun Zhu and Pengfeng Xiao and Feng Xue-zhi and Wang Zuo and Jiang Luyuan},
  title = {{Multi-scale accuracy assessment of land cover datasets based on histo-variograms}},
  booktitle = {National},
  year = {2013},
  doi = {10.11834/jrs.20133041},
}

Loading related content...

Figure