Tree species classification from hyperspectral data using graph-regularized neural networks

Debmita Bandyopadhyay, Subhadip MukherjeearXiv 2022 · DOI · OpenReview

TL;DR

This paper introduces a novel Graph-Regularized Neural Network (GRNN) for tree species classification using hyperspectral imagery. GRNN effectively combines pixel-level neural network predictions with superpixel-level graph regularization, achieving high accuracy even with extremely sparse labeling (<1% of pixels). This approach produces realistic, tree-crown-scale classification maps, outperforming existing methods on challenging datasets.

What Problem Does This Paper Address?

Classifying tree species from hyperspectral imagery is crucial for forest management and conservation. However, this task is challenging due to spectral similarities between species and the difficulty of obtaining accurate, large-scale ground truth labels, especially in remote or inaccessible areas like tropical forests. Traditional supervised methods struggle with limited labels, while deep learning models can produce accurate but spatially unrealistic maps, particularly for unlabeled regions. The paper aims to bridge this gap by developing a method that leverages both spectral-spatial information and sparse labels to generate accurate and ecologically meaningful classification maps.

Methods

The proposed Graph-Regularized Neural Network (GRNN) algorithm integrates several key components:

  • Superpixel Segmentation: The Simple Linear Iterative Clustering (SLIC) algorithm is used to segment the hyperspectral image into superpixels. These superpixels represent more coherent regions, often approximating tree crowns, and form the nodes of a graph.
  • Pixel-wise Neural Network (NN) Classifier: A fully-connected neural network is trained to classify individual pixels. This network learns spectral-spatial features and provides initial probability maps for each class.
  • Graph Construction: A graph is constructed where nodes are the superpixels. Edge weights between superpixels represent their pairwise similarity based on extracted features. This graph captures the spatial relationships between superpixels.
  • Graph Regularization: The core innovation is a graph energy function that regularizes the NN's output. This function encourages smoothness in the predicted superpixel probability maps, ensuring that adjacent superpixels are likely to have similar classifications. This is achieved by minimizing a loss function that includes terms for pixel-wise classification accuracy, superpixel-wise classification accuracy, graph smoothness, and intra-superpixel variance.
  • Label Propagation: High-confidence predictions from the trained NN are used to augment the limited ground-truth labels. A classical semi-supervised label propagation algorithm is then applied on the superpixel graph using these augmented labels to generate the final, smoothed classification map.

Datasets

The paper uses two datasets:

  • Indian Pines (IP) HSI: A standard benchmark dataset for hyperspectral image classification, collected over an agricultural site with 16 classes. This dataset is used for performance comparison with existing state-of-the-art methods.
  • French Guiana (FG) HSI: A newly collected, high-resolution, and highly heterogeneous dataset from a tropical rainforest. This dataset presents a more challenging scenario with limited ground-truth labels (<1% of pixels labeled) and significant class imbalance, making it ideal for testing the robustness of the proposed method.

Key Insights and Ideas

  • Bridging Pixel and Superpixel Information: The GRNN effectively combines the strengths of pixel-level spectral analysis (via the NN) with superpixel-level spatial context (via the graph). This hybrid approach is key to achieving both accuracy and realism.
  • The Power of Sparse Labels: The method demonstrates remarkable performance even with extremely limited labeled data (<1% of pixels), highlighting its suitability for real-world scenarios where extensive annotation is infeasible.
  • Graph Regularization for Realism: The graph-based regularization term is crucial for generating spatially coherent and ecologically realistic classification maps, preventing the "checkerboard" effect often seen in purely pixel-based methods.
  • Robustness to Label Variation: GRNN shows minimal degradation in accuracy when the number of training samples is varied or when labels are randomly sampled, indicating its robustness and reliability.

Key Results

  • Indian Pines Dataset: GRNN achieved the highest overall accuracy (OA) and Cohen's kappa (κ) coefficient among the compared methods, outperforming even the closest competitor, SGL (a superpixel-based semi-supervised method), by approximately 0.08 in κ.
  • French Guiana Dataset: With less than 1% of pixels labeled, GRNN achieved a κ of 0.79, demonstrating its effectiveness on a challenging, heterogeneous tropical forest dataset. The accuracy improved to 0.89 with more labeled data.
  • Robustness: GRNN exhibited significantly smaller standard deviations in accuracy across different trials and training sample sizes compared to SVM and SGL, indicating greater stability.
  • Qualitative Improvement: The classification maps produced by GRNN are visibly smoother and more realistic, better capturing the tree crown structures compared to SVM and SGL.

Limitations and Future Work

  • Hyperparameter Tuning: The paper acknowledges that the selection of hyperparameters for both graph construction and GRNN training is crucial and was performed through cross-validation on a small validation set. A more principled approach to learning these hyperparameters directly from the data is suggested as future work.
  • Automatic Feature Learning: The authors propose exploring automatic feature learning from extracted superpixels, rather than relying on handcrafted features, to further enhance GRNN's performance.
  • Comparison with Deep Learning: A more extensive comparison with other deep learning-based state-of-the-art classification methods is planned for future studies.

Summary updated

Abstract

We propose a novel graph-regularized neural network (GRNN) algorithm for tree species classification. The proposed algorithm encompasses superpixel-based segmentation for graph construction, a pixel-wise neural network classifier, and the label propagation technique to generate an accurate and realistic (emulating tree crowns) classification map on a sparsely annotated data set. GRNN outperforms several state-of-the-art techniques not only for the standard Indian Pines HSI but also achieves a high classification accuracy (approx. 92%) on a new HSI data set collected over the heterogeneous forests of French Guiana (FG) when less than 1% of the pixels are labeled. We further show that GRNN is competitive with the state-of-the-art semi-supervised methods and exhibits a small deviation in accuracy for different numbers of training samples and over repeated trials with randomly sampled labeled pixels for training.
BibTeX
@misc{bandyopadhyay2022tree,
  author = {Debmita Bandyopadhyay and Subhadip Mukherjee},
  title = {{Tree species classification from hyperspectral data using graph-regularized neural networks}},
  year = {2022},
  eprint = {2208.08675},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2208.08675},
}
Figure