Solving Spectrum Unmixing as a Multi-Task Bayesian Inverse Problem with Latent Factors for Endmember Variability

Dong Wu, Mingmin Chi, Xuan Zang, Bo PengAAAI 2024 · DOI

Abstract

Spectral unmixing is a cornerstone of hyperspectral imaging, yet its accuracy is frequently compromised by Endmember Variability (EV)—the phenomenon where material spectral signatures shift due to environmental or intrinsic factors. This paper introduces BIPU, a novel framework that treats unmixing as a multi-task Bayesian inverse problem. BIPU utilizes latent factor modeling to account for variability and incorporates a classification-based "abundance mask" to constrain the solution space. Evaluated on a massive new textile dataset (FENIR) and standard remote sensing benchmarks, BIPU demonstrates state-of-the-art performance, significantly reducing reconstruction errors and improving estimation stability.

Introduction: The Challenge of Endmember Variability

In an ideal hyperspectral unmixing scenario, every material possesses a unique, fixed "spectral fingerprint." In practice, however, factors such as changing illumination, atmospheric conditions, and intrinsic material differences (e.g., varying textile weaves) cause these signatures to fluctuate. This is known as Endmember Variability (EV).

Visualizing Endmember Variability (EV) across different material spectra.

Traditional unmixing models often fail because they assume fixed endmembers, leading to mismatched abundance ratios. While deep learning has been applied to this problem, it often struggles with its ill-posed nature—multiple combinations of varying signatures can result in the same observed pixel. Existing neural network approaches frequently lack effective convergence constraints and robust uncertainty quantification.

Methodology: The BIPU Framework

The researchers introduce the Bayesian Inverse Problem for Unmixing (BIPU) to address these limitations through a multi-task learning architecture built on four key pillars:

1. Latent Factor Modeling

Instead of predicting a single abundance value, BIPU models both abundance ($a$) and variability ($v$) as latent distributions using a Variational Autoencoder (VAE) style structure. This probabilistic approach allows the model to better handle the inherent uncertainty in spectral signatures.

Comparison between a standard VAE architecture and the proposed BIPU framework.

2. Multi-Task Learning with an Abundance Mask

A critical innovation in BIPU is the simultaneous performance of multi-label classification. The model generates an abundance mask that identifies which materials are likely present in a pixel. By "zeroing out" non-existent materials, the mask significantly shrinks the search space, preventing the model from assigning spurious abundances to irrelevant endmembers and ensuring more stable convergence.

3. Total Correlation (TC) Loss

To maintain the integrity of the latent factors, the authors apply a Total Correlation penalty. This forces the abundance and variability distributions to remain independent, ensuring that the variability factor ($v$) captures only spectral shifts and not material quantity.

4. Laplace Approximation

BIPU employs Laplace approximation within its Bayesian framework to quantify and minimize uncertainty during the reconstruction process, providing a principled way to navigate the ill-posed landscape of the inverse problem.

Datasets and Experimental Setup

The paper evaluates BIPU across four diverse datasets:

  • FENIR (New): A large-scale near-infrared textile fiber dataset containing nearly 400,000 spectra across 12 fiber components, reflecting real-world variability.
  • Jasper Ridge & EnMAP Munish: Standard remote sensing hyperspectral scenes.
  • Synthetic Dataset: Generated from the USGS spectral library.

Key Results and Insights

BIPU demonstrated significant improvements over eight baseline models, including specialized unmixing networks like EGU-Net and TANet:

  • Superior Accuracy: On the FENIR dataset, BIPU achieved an 18.6% to 31.48% reduction in aRMSE (average Root Mean Square Error) compared to baselines.
  • Enhanced Stability: The model showed a marked increase in $\gamma$-accuracy, reflecting its stability across different noise levels and variations.
  • Generalization: On the Jasper Ridge dataset, BIPU achieved the lowest error rates (aRMSE of 7.02 vs. the next best baseline at 7.54).

Conclusion

BIPU addresses the long-standing problem of Endmember Variability by combining Bayesian inference with multi-task learning. The introduction of the abundance mask and the decoupling of latent factors via Total Correlation loss prove to be highly effective strategies for solving the ill-posed spectral unmixing problem. Future work will explore more complex non-linear encoders and aim to make the latent variability factors more physically interpretable.

Reference: Wu, D., Chi, M., Zang, X., & Peng, B. (2024). Solving Spectrum Unmixing as a Multi-Task Bayesian Inverse Problem with Latent Factors for Endmember Variability. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI-24).

Summary updated

BibTeX
@inproceedings{wu2024solving,
  author = {Dong Wu and Mingmin Chi and Xuan Zang and Bo Peng},
  title = {{Solving Spectrum Unmixing as a Multi-Task Bayesian Inverse Problem with Latent Factors for Endmember Variability}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i14.29518},
}
Figure