CoSpace: Common Subspace Learning from Hyperspectral-Multispectral Correspondences

Danfeng Hong, Naoto Yokoya, Jocelyn Chanussot, Xiao Xiang Zhu

Abstract

With a large amount of open satellite multispectral imagery (e.g., Sentinel-2 and Landsat-8), considerable attention has been paid to global multispectral land cover classification. However, its limited spectral information hinders further improving the classification performance. Hyperspectral imaging enables discrimination between spectrally similar classes but its swath width from space is narrow compared to multispectral ones. To achieve accurate land cover classification over a large coverage, we propose a cross-modality feature learning framework, called common subspace learning (CoSpace), by jointly considering subspace learning and supervised classification. By locally aligning the manifold structure of the two modalities, CoSpace linearly learns a shared latent subspace from hyperspectral-multispectral (HS-MS) correspondences. The multispectral out-of-samples can be then projected into the subspace, which are expected to take advantages of rich spectral information of the corresponding hyperspectral data used for learning, and thus leads to a better classification. Extensive experiments on two simulated HS-MS datasets (University of Houston and Chikusei), where HS-MS data sets have trade-offs between coverage and spectral resolution, are performed to demonstrate the superiority and effectiveness of the proposed method in comparison with previous state-of-the-art methods.

Introduction: The Spectral-Spatial Trade-off

In the field of remote sensing, researchers face a persistent trade-off between spectral resolution and spatial coverage. Multispectral (MS) satellites, such as Sentinel-2, provide broad, global coverage but possess a limited number of spectral bands. This lack of spectral depth often makes it difficult to distinguish between materials with similar signatures, such as different types of vegetation or complex urban surfaces. Conversely, hyperspectral (HS) sensors capture hundreds of narrow spectral bands, allowing for precise material identification, but they typically have a very narrow swath width, limiting their use for large-scale mapping.

CoSpace addresses this "cross-modal feature learning" challenge by asking: Can we use a small patch of high-fidelity hyperspectral data to "teach" a global multispectral model to see more effectively? The goal is to transfer the discriminative knowledge from the HS domain into the MS domain, enabling more accurate large-scale land-cover classification using only MS data at the testing stage.

Methodology

The proposed CoSpace (Common Subspace Learning) framework is an alignment-based approach designed to bridge the gap between these two modalities. Unlike traditional fusion methods that require both HS and MS data for every pixel, CoSpace learns a mapping from a small overlapping area and applies it to wide-area MS data.

The CoSpace framework workflow, illustrating the training phase on overlapping areas and the testing phase on large-scale multispectral data.

Key technical components of the framework include:

  • Shared Latent Subspace: CoSpace learns a linear mapping that projects both HS and MS data into a common coordinate system where their features become directly comparable.
  • Joint Learning and Optimization: Rather than treating subspace learning and classification as separate steps, CoSpace optimizes them simultaneously. This ensures the learned subspace is specifically tailored to maximize classification accuracy.
  • Manifold Alignment: The framework utilizes a joint graph structure (a Laplacian matrix) to ensure that pixels belonging to the same class—regardless of the sensor source—remain close to each other in the latent subspace.
  • ADMM Optimization: To solve the complex joint optimization problem, the authors utilize a customized algorithm based on the Alternating Direction Method of Multipliers (ADMM), ensuring efficient convergence.

Key Insights: Feature Transfer and Alignment

A critical insight of this work is that discriminative power is transferable across modalities. Even if an MS sensor cannot natively "see" a specific chemical signature, CoSpace identifies patterns in the MS bands that correlate with those signatures in the HS domain.

Comparison between fusion-based (FJFL) and alignment-based (ASFL/CoSpace) feature learning, highlighting the ability to handle out-of-samples.

By incorporating label information directly into the subspace learning process (supervised alignment), CoSpace ensures that the resulting "bridge" between modalities is optimized for distinguishing between classes. Qualitatively, the model has also shown resilience in shadowed areas, which are notoriously difficult for standard multispectral classification methods.

Experimental Results

The researchers evaluated CoSpace on two major simulated benchmarks, using Sentinel-2 spectral response filters to create MS data from HS sources:

  1. University of Houston Dataset: Covering an urban campus with 144 spectral bands at 2.5m resolution.
  2. Chikusei Dataset: Covering agricultural and urban areas in Japan with 128 spectral bands at 2.5m resolution.

In both cases, CoSpace consistently outperformed the MS-only baseline and several state-of-the-art manifold alignment techniques. On the Houston dataset, CoSpace improved Overall Accuracy (OA) by approximately 7% to 10%. On the Chikusei dataset, it achieved an OA of 75.69%, compared to the 71.11% baseline. Notably, the model showed significant gains in difficult classes like Residential and Railway, which are frequently confused in standard MS imagery.

Conclusion and Future Directions

While CoSpace represents a significant step forward, the authors identify two primary areas for future research. First, the current model is linear; exploring non-linear relationships through Deep Learning could further enhance performance on heterogeneous data. Second, the model currently assumes that the training area contains all classes present in the larger test scene. Developing methods to handle "unseen" classes in the MS-only regions remains a key challenge for global-scale application.

Summary updated

Abstract

With a large amount of open satellite multispectral imagery (e.g., Sentinel-2 and Landsat-8), considerable attention has been paid to global multispectral land cover classification. However, its limited spectral information hinders further improving the classification performance. Hyperspectral imaging enables discrimination between spectrally similar classes but its swath width from space is narrow compared to multispectral ones. To achieve accurate land cover classification over a large coverage, we propose a cross-modality feature learning framework, called common subspace learning (CoSpace), by jointly considering subspace learning and supervised classification. By locally aligning the manifold structure of the two modalities, CoSpace linearly learns a shared latent subspace from hyperspectral-multispectral(HS-MS) correspondences. The multispectral out-of-samples can be then projected into the subspace, which are expected to take advantages of rich spectral information of the corresponding hyperspectral data used for learning, and thus leads to a better classification. Extensive experiments on two simulated HSMS datasets (University of Houston and Chikusei), where HS-MS data sets have trade-offs between coverage and spectral resolution, are performed to demonstrate the superiority and effectiveness of the proposed method in comparison with previous state-of-the-art methods.
BibTeX
@article{hong2019cospace,
  author = {Danfeng Hong and Naoto Yokoya and Jocelyn Chanussot and Xiao Xiang Zhu},
  title = {{CoSpace: Common Subspace Learning from Hyperspectral-Multispectral Correspondences}},
  journal = {IEEE Trans. Geosci. Remote. Sens.},
  year = {2019},
  url = {https://openreview.net/forum?id=YRrPY4uDLj},
}
Figure