CoSpace: Common Subspace Learning From Hyperspectral-Multispectral Correspondences

Danfeng Hong, Naoto Yokoya, Jocelyn Chanussot, Xiao Xiang Zhu

TL;DR

CoSpace is a novel machine learning framework that allows wide-coverage multispectral (MS) satellite data to "inherit" the superior discriminative power of narrow-swath hyperspectral (HS) imagery. By learning a shared latent subspace from small areas where both sensors overlap, CoSpace significantly improves land-cover classification accuracy across vast regions where only MS data is available.

What Problem Does This Paper Address?

In remote sensing, we face a frustrating trade-off. Multispectral satellites like Sentinel-2 provide "big picture" coverage of the Earth but have low spectral resolution (few color bands), making it hard to distinguish between similar materials (e.g., different types of grass or urban surfaces). Hyperspectral sensors provide "rich detail" with hundreds of bands, allowing for precise identification, but they only see tiny slivers of the planet at a time.

The authors tackle a "cross-modal feature learning" challenge: Can we use a small patch of expensive hyperspectral data to teach a global multispectral model how to see better? The goal is to transfer the high-resolution "knowledge" from the HS domain into the MS domain so that large-scale mapping becomes more accurate.

Methods

The authors propose CoSpace (Common Subspace Learning). Unlike traditional "fusion" methods that require both types of data for every pixel, CoSpace is an alignment-based approach.

Key technical components include:

  • Shared Latent Subspace: It learns a linear mapping that projects both HS and MS data into a new, common coordinate system where their features are comparable.
  • Joint Learning: Standard methods often find a subspace first and then train a classifier. CoSpace does both simultaneously. It optimizes the subspace specifically to make the final classification easier.
  • Manifold Alignment: It uses a joint graph structure (a Laplacian matrix) to ensure that pixels belonging to the same class—regardless of which sensor they came from—stay close together in the new subspace.
  • ADMM Optimization: To solve the complex math behind this joint optimization, the authors developed a customized algorithm based on the Alternating Direction Method of Multipliers (ADMM).

Datasets

The researchers evaluated CoSpace on two major simulated benchmarks:

  1. University of Houston Dataset: 144 spectral bands at 2.5m resolution, covering an urban campus environment.
  2. Chikusei Dataset: 128 spectral bands at 2.5m resolution, covering agricultural and urban areas in Japan.

In both cases, the authors simulated Multispectral data by passing the Hyperspectral data through the spectral response filters of the Sentinel-2 satellite. They specifically designed the experiments so that training happened only on small overlapping areas, while testing was performed on the much larger "MS-only" regions.

Key Insights and Ideas

  • The "Aha" Moment - Feature Transfer: The most important finding is that discriminative power is transferable. Even if an MS sensor can't "see" a specific chemical signature, CoSpace finds the patterns in the MS bands that correlate with that signature in the HS domain.
  • Supervision is Secret Sauce: Unsupervised alignment (which just looks at data shapes) struggles. By including label information directly into the subspace learning process, CoSpace ensures the "bridge" it builds between modalities is actually useful for distinguishing classes.
  • Shadow Resilience: Qualitatively, the model proved surprisingly good at identifying materials in shadowed areas—a notorious "blind spot" for standard multispectral classification.

Key Results

CoSpace consistently outperformed the "Baseline" (using MS data alone) and several state-of-the-art dimensionality reduction and manifold alignment techniques (like P-JDR and L-SMA).

  • Accuracy Boost: On the Houston dataset, CoSpace improved Overall Accuracy (OA) by roughly 7% to 10% over the baseline, depending on the classifier used (Nearest Neighbor or SVM).
  • Chikusei Performance: It achieved an OA of 75.69%, significantly higher than the MS baseline of 71.11%.
  • Class-Specific Gains: The model showed dramatic improvements in "difficult" classes like Residential and Railway, which are often confused in standard multispectral imagery.

Limitations and Future Work

The authors acknowledge two main hurdles for the future:

  1. Linearity: CoSpace currently uses linear modeling. While efficient, it might struggle with highly complex, "heterogeneous" data where the relationship between sensors is non-linear. The authors suggest exploring Deep Learning to create a non-linear version of CoSpace.
  2. Class Consistency: The model assumes that the small HS training area contains all the classes found in the large MS scene. In the real world, a large MS image might contain new classes not seen in the HS training patch. Generalizing the model to handle these "unseen" classes is a key next step.

Summary updated

Abstract

With a large amount of open satellite multispectral (MS) imagery (e.g., Sentinel-2 and Landsat-8), considerable attention has been paid to global MS land cover classification. However, its limited spectral information hinders further improving the classification performance. Hyperspectral imaging enables discrimination between spectrally similar classes but its swath width from space is narrow compared to MS ones. To achieve accurate land cover classification over a large coverage, we propose a cross-modality feature learning framework, called common subspace learning (CoSpace), by jointly considering subspace learning and supervised classification. By locally aligning the manifold structure of the two modalities, CoSpace linearly learns a shared latent subspace from hyperspectral-MS (HS-MS) correspondences. The MS out-of-samples can be then projected into the subspace, which are expected to take advantages of rich spectral information of the corresponding hyperspectral data used for learning, and thus leads to a better classification. Extensive experiments on two simulated HS-MS data sets (University of Houston and Chikusei), where HS-MS data sets have tradeoffs between coverage and spectral resolution, are performed to demonstrate the superiority and effectiveness of the proposed method in comparison with previous state-of-the-art methods.
BibTeX
@article{hong2019cospace,
  author = {Danfeng Hong and Naoto Yokoya and Jocelyn Chanussot and Xiao Xiang Zhu},
  title = {{CoSpace: Common Subspace Learning From Hyperspectral-Multispectral Correspondences}},
  journal = {IEEE Trans. Geosci. Remote. Sens.},
  year = {2019},
  url = {https://openreview.net/forum?id=YRrPY4uDLj},
}

Loading related content...

Figure