A Comparison of Machine Learning Algorithms for Mapping of Complex Surface-Mined and Agricultural Landscapes Using ZiYuan-3 Stereo Satellite Imagery

Xianju Li, Weitao Chen, Xinwen Cheng, Lizhe WangRemote 2016 · DOI

TL;DR

This study tackles the challenging task of mapping complex landscapes that combine surface mining and agriculture using high-resolution satellite imagery. By introducing novel features and a feature selection method, the researchers significantly improved classification accuracy and reduced data dimensionality, demonstrating the effectiveness of their approach for environmental monitoring and land management.

What Problem Does This Paper Address?

Mapping land cover (LCM) is crucial for understanding environmental changes, but it becomes particularly difficult in complex landscapes where different land types are spectrally similar or have intricate spatial arrangements. The authors focus on a specific challenge: mixed surface-mined and agricultural landscapes (CSMAL). These areas present unique difficulties due to the presence of mining-related features (e.g., open pits, waste piles, processing plants) alongside agricultural fields, often with overlapping spectral signatures. Existing research has largely focused on either mining or agricultural landscapes separately, leaving a gap in understanding how to effectively map these combined, complex environments using high spatial resolution imagery.

Methods

The core of this research lies in a novel approach that combines:

  • ZiYuan-3 (ZY-3) Stereo Satellite Imagery: This provides both high-resolution spectral bands and the capability to derive topographic data (Digital Terrain Models - DTMs).
  • Novel Feature Engineering: The authors introduced new features, including mean and standard deviation filters applied to spectral bands, and topographic features derived from the ZY-3 DTMs. They also utilized standard features like spectral bands, Vegetation Index (NDVI), and Principal Component (PC) bands.
  • Wrapper Feature Selection (FS): A wrapper method (varSelRF) was employed to intelligently select the most informative features from the extensive set generated. This is crucial for reducing redundancy and mitigating the "curse of dimensionality."
  • Machine Learning Algorithms (MLAs): Three popular MLAs were compared: Random Forest (RF), Support Vector Machine (SVM), and Artificial Neural Network (ANN).

The study performed two main tasks: mapping the entire mixed landscape (MSMAL) and classifying only the surface-mined land within it (CSML).

Datasets

The primary dataset used was ZiYuan-3 (ZY-3) stereo satellite imagery acquired on June 20, 2012, covering an area in Wuhan City, China. This imagery provided both multispectral and panchromatic bands, as well as stereo capabilities for DTM generation.

Field surveys and visual interpretation of the ZY-3 imagery were used to collect ground truth data for training and testing. The study also mentions the use of existing national land cover standards (GB/T21010-2007) as a reference for developing the land cover classification schemes. No new benchmark datasets were introduced, but the study area itself represents a complex landscape that differs from those typically studied.

Key Insights and Ideas

  • Novel Features Matter: The introduction of mean and standard deviation filters on spectral bands, along with topographic features derived from stereo imagery, significantly contributed to classification accuracy, especially for the complex mining areas.
  • Texture is Overrated (in this context): Surprisingly, texture measures, often used in LCM, provided little to no useful information for classification in this specific complex landscape. This highlights that feature effectiveness is highly context-dependent.
  • Feature Selection is Crucial: The wrapper FS method was highly effective, substantially reducing the feature set (by up to 87% for CSML) while often improving classification accuracy. This demonstrates the power of intelligent feature selection in complex scenarios.
  • MLAs Have Different Strengths and Sensitivities: RF and SVM generally outperformed ANN, especially when using feature subsets. Notably, SVM was found to be the most sensitive to feature selection, while RF was the least sensitive.

Key Results

  • Improved Accuracy with FS: Feature selection led to an average accuracy improvement of 4.48% for MSMAL and 11.39% for CSML (with RF and SVM).
  • Reduced Feature Sets: FS reduced the feature set by 68% for MSMAL and 87% for CSML.
  • Top Performing Algorithms:
    • For MSMAL (feature subset): RF achieved the highest overall accuracy (77.57%), followed by SVM (72.00%) and ANN (64.29%).
  • For CSML (feature subset): SVM achieved the highest overall accuracy (87.34%), followed by RF (87.18%) and ANN (71.88%).
  • Statistically Significant Differences: McNemar tests confirmed statistically significant differences in classification accuracy between the MLAs and between feature subset-based and all feature-based models (with some exceptions for ANN).
  • Feature Importance: DTM was identified as the most important feature for MSMAL, while NDVI was paramount for CSML. Red and NIR spectral bands, along with specific filter features (mean filter with larger kernels), also showed high importance.

Limitations and Future Work

  • Texture Ineffectiveness: The study found texture measures to be of little use, suggesting that their effectiveness is application-specific.
  • ANN Performance Sensitivity: The ANN algorithm's performance was sensitive to the test set size, particularly for MSMAL.
  • Future Directions: The authors suggest exploring:
  • Other effective texture measures derived from stereo imagery.
  • Addressing class imbalance issues.
  • Investigating the effect of different test set sizes on accuracy assessment.
  • Object-based fine mapping of the second-level land cover scheme.
  • Comparing and integrating various feature reduction methods.
  • Developing ensemble learning algorithms for similar complex landscapes.

Summary updated

Abstract

Land cover mapping (LCM) in complex surface-mined and agricultural landscapes could contribute greatly to regulating mine exploitation and protecting mine geo-environments. However, there are some special and spectrally similar land covers in these landscapes which increase the difficulty in LCM when employing high spatial resolution images. There is currently no research on these mixed complex landscapes. The present study focused on LCM in such a mixed complex landscape located in Wuhan City, China. A procedure combining ZiYuan-3 (ZY-3) stereo satellite imagery, the feature selection (FS) method, and machine learning algorithms (MLAs) (random forest, RF; support vector machine, SVM; artificial neural network, ANN) was proposed and first examined for both LCM of surface-mined and agricultural landscapes (MSMAL) and classification of surface-mined land (CSML), respectively. The mean and standard deviation filters of spectral bands and topographic features derived from ZY-3 stereo images were newly introduced. Comparisons of three MLAs, including their sensitivities to FS and whether FS resulted in significant influences, were conducted for the first time in the present study. The following conclusions are drawn. Textures were of little use, and the novel features contributed to improve classification accuracy. Regarding the influence of FS: FS substantially reduced feature set (by 68% for MSMAL and 87% for CSML), and often improved classification accuracies (with an average value of 4.48% for MSMAL using three MLAs, and 11.39% for CSML using RF and SVM); FS showed statistically significant improvements except for ANN-based MSMAL; SVM was most sensitive to FS, followed by ANN and RF. Regarding comparisons of MLAs: for MSMAL based on feature subset, RF achieved the greatest overall accuracy of 77.57%, followed by SVM and ANN; for CSML, SVM had the highest accuracies (87.34%), followed by RF and ANN; based on the feature subsets, significant differences were observed for MSMAL and CSML using any pair of MLAs. In general, the proposed approach can contribute to LCM in complex surface-mined and agricultural landscapes.
BibTeX
@inproceedings{li2016comparison,
  author = {Xianju Li and Weitao Chen and Xinwen Cheng and Lizhe Wang},
  title = {{A Comparison of Machine Learning Algorithms for Mapping of Complex Surface-Mined and Agricultural Landscapes Using ZiYuan-3 Stereo Satellite Imagery}},
  booktitle = {Remote},
  year = {2016},
  doi = {10.3390/rs8060514},
}
Figure