Locality and Structure Regularized Low Rank Representation for Hyperspectral Image Classification

Qi Wang, Xiang He, Xuelong Li

TL;DR

This paper introduces a Locality and Structure Regularized Low Rank Representation (LSLRR) model for Hyperspectral Image (HSI) classification. By combining a novel spatial-spectral distance metric with a structure-preserving strategy, the model forces data into a highly discriminative, block-diagonal format, significantly outperforming traditional Low Rank Representation (LRR) and standard SVMs, especially when training data is limited.

What Problem Does This Paper Address?

Hyperspectral images (HSIs) capture hundreds of continuous spectral bands per pixel, providing rich information about land materials. However, classifying these pixels is notoriously difficult due to the "Hughes phenomenon" (where high dimensionality degrades classifier performance) and a chronic lack of labeled training data.

To solve this, researchers often use Low Rank Representation (LRR), a technique that assumes high-dimensional data actually lies in multiple low-dimensional subspaces. While LRR is great at capturing the global structure of data, standard LRR has two major blind spots:

  1. It ignores local spatial context: It treats pixels as independent, ignoring the fact that physically neighboring pixels usually belong to the same class.
  2. It lacks discriminative structure: Ideally, a representation matrix should be "block-diagonal" (where samples of Class A only correlate with Class A, Class B with Class B, etc.). Standard LRR struggles to form this clean separation, making the final classification step harder.

Methods

The authors propose LSLRR, which upgrades standard LRR with three key innovations:

  • Locality Constraint Criterion (LCC): The authors introduce a new distance metric that fuses spectral features (the pixel's actual data) with spatial features (the pixel's physical coordinates). A hyperparameter ($m$) balances the two. This metric acts as a penalty term during optimization, forcing the model to respect the local geometric structure of the image.
  • Structure Preserving Strategy (SPS): To fix the "messy matrix" problem, the authors introduce an auxiliary matrix that forces the learned representation into a class-wise block-diagonal structure. By utilizing the labels of the training data and a Gaussian similarity function for the testing data, the model strictly groups similar classes together.
  • Dictionary Learning: Standard LRR uses all data samples as its "dictionary" to reconstruct the image, which is computationally heavy and introduces noise. LSLRR instead iteratively learns a compact, discriminative dictionary from the data, filtering out corrupted samples.

The resulting complex objective function is solved using the high-efficiency Inexact Augmented Lagrange Multiplier (IALM) algorithm.

Datasets

The model was evaluated on three standard, publicly available remote sensing benchmarks:

  • Indian Pines: An agricultural region with 200 spectral bands and 16 classes.
  • Pavia University: An urban area with 103 spectral bands and 9 classes.
  • Salinas: An agricultural/vegetation scene with 204 spectral bands and 16 classes.

Key Insights and Ideas

  • Spatial context is a cheat code for HSI: Relying solely on spectral signatures leads to noisy, "salt-and-pepper" classification maps. Forcing the model to acknowledge the physical distance between pixels drastically smooths and corrects predictions.
  • Block-diagonal structures eliminate the need for complex downstream classifiers: Because the SPS forces the representation matrix into a clean block-diagonal form, the model doesn't need a heavy classifier (like an SVM or Neural Network) to figure out the labels. It simply looks at which block a test pixel's representation falls into and assigns the label via a basic summation.
  • Distance metrics must adapt to geography: The parameter $m$ (balancing spatial vs. spectral distance) must be tuned to the physical layout of the scene. Urban areas (Pavia) with slender, scattered features require a lower $m$, while uniform, blocky agricultural fields (Salinas) benefit from a higher $m$.

Key Results

  • Massive Baseline Improvements: LSLRR absolutely crushed the baseline LRR model. On the Indian Pines dataset, LSLRR achieved an Overall Accuracy (OA) of 95.63%, compared to standard LRR's 70.47%—a massive >25% jump.
  • State-of-the-Art Performance: Across the board, LSLRR outperformed advanced competitors like SVM with Composite Kernels (SVMCK) and Joint Robust Sparse Representation (JRSRC). It achieved 98.52% OA on Pavia University and 97.77% OA on Salinas.
  • Robustness to Limited Data: The model maintained its superior accuracy even when the training data was restricted to just 5% or 10% of the total labeled pixels.
  • Computational Cost: While slower than a basic SVM (336 seconds vs 4 seconds on Indian Pines), LSLRR is computationally competitive with other advanced sparse/low-rank methods (e.g., LGIDL took 382 seconds).

Limitations and Future Work

While highly accurate, the paper reveals a few limitations. First, the computational complexity (running time) is significantly higher than simple pixel-wise classifiers, making it less ideal for real-time applications. Second, the model's success heavily relies on the manual tuning of several hyperparameters ($\alpha$, $\beta$, $m$, $\lambda$) via cross-validation, which must be uniquely calibrated to the spatial compactness of every new dataset. Future work would likely benefit from automated or adaptive parameter selection mechanisms.

Summary updated

Abstract

Hyperspectral image (HSI) classification, which aims to assign an accurate label for hyperspectral pixels, has drawn great interest in recent years. Although low-rank representation (LRR) has been used to classify HSI, its ability to segment each class from the whole HSI data has not been exploited fully yet. LRR has a good capacity to capture the underlying low-dimensional subspaces embedded in original data. However, there are still two drawbacks for LRR. First, the LRR does not consider the local geometric structure within data, which makes the local correlation among neighboring data easily ignored. Second, the representation obtained by solving LRR is not discriminative enough to separate different data. In this paper, a novel locality- and structure-regularized LRR (LSLRR) model is proposed for HSI classification. To overcome the above-mentioned limitations, we present locality constraint criterion and structure preserving strategy to improve the classical LRR. Specifically, we introduce a new distance metric, which combines both spatial and spectral features, to explore the local similarity of pixels. Thus, the global and local structures of HSI data can be exploited sufficiently. In addition, we propose a structural constraint to make the representation have a near-block-diagonal structure. This helps to determine the final classification labels directly. Extensive experiments have been conducted on three popular HSI data sets. And the experimental results demonstrate that the proposed LSLRR outperforms other state-of-the-art methods.
BibTeX
@article{wang2019locality,
  author = {Qi Wang and Xiang He and Xuelong Li},
  title = {{Locality and Structure Regularized Low Rank Representation for Hyperspectral Image Classification}},
  journal = {IEEE Trans. Geosci. Remote. Sens.},
  year = {2019},
  url = {https://openreview.net/forum?id=xF4qVlqlpFs},
}

Loading related content...

Figure