Model-Based Deep Autoencoder Networks for Nonlinear Hyperspectral Unmixing

Haoqing Li, Ricardo Augusto Borsoi, Tales Imbiriba, Pau Closas, José Carlos Moreira Bermudez, Deniz ErdogmusIEEE GRSL 2021 · DOI · OpenReview

TL;DR

This paper introduces a novel "model-based" deep autoencoder (AEC) architecture for hyperspectral unmixing (HU) that explicitly leverages the inverse relationship between the encoder and decoder. By incorporating prior knowledge about the mixing process into the AEC's structure, the authors achieve more robust and accurate nonlinear HU, outperforming existing methods on synthetic and real-world datasets.

What Problem Does This Paper Address?

Hyperspectral imaging captures detailed spectral information across numerous contiguous bands, allowing for the identification and quantification of materials within a scene. Hyperspectral unmixing (HU) is the process of determining the spectral signatures of pure materials (endmembers) and their fractional abundances within each pixel. While linear mixing models (LMMs) are a common starting point, real-world scenarios often involve complex nonlinear interactions between materials, necessitating nonlinear unmixing (NMU) techniques.

Existing unsupervised NMU methods, particularly those based on autoencoders (AECs), have shown promise but often treat the encoder and decoder as independent neural networks. This paper argues that this overlooks a crucial aspect: the encoder should ideally act as the inverse of the decoder (which models the mixing process). Failing to enforce this relationship can lead to less robust and less interpretable results, especially when dealing with complex nonlinearities. The authors aim to bridge this gap by developing an AEC that explicitly incorporates this inverse relationship.

Methods

The core innovation lies in the proposed Model-based AutoenCoder for hyperspectral Unmixing (MAC-U). The authors model the nonlinear mixing process as a linear mixture with an added nonlinear fluctuation, represented by equation (3): $Y_n = M a_n + \psi(M, a_n) + e_n$.

The MAC-U architecture consists of:

  • Decoder ($h$): This network models the nonlinear mixing process. It's designed with an additive linear and nonlinear part, similar to prior work, where the linear part is associated with endmember signatures ($M$) and the nonlinear part takes abundances ($a_n$) and $M$ as input. This allows for flexible modeling of arbitrary mixtures.
  • Encoder ($g$): This is where the novelty lies. Instead of a generic nonlinear network, the encoder is structured to approximate the inverse of the decoder. Specifically, it's designed to be related to the pseudoinverse of the endmember matrix ($M^\dagger$). This constraint is incorporated through a specific architecture involving a linear term related to $M^\dagger$ and a nonlinear term, balanced by a learnable parameter $\alpha$.

The training objective is a cost function that minimizes the reconstruction error, regularizes the nonlinear parts of the encoder and decoder, and enforces a constraint related to the pseudoinverse of the endmember matrix.

Datasets

The paper evaluates the proposed MAC-U method on:

  • Synthetic Data: Two datasets, Data Cube 1 (DC1) and Data Cube 2 (DC2), were generated. These datasets featured 3 endmembers from the USGS Spectral Library with 224 bands. Abundance maps were sampled from a Dirichlet distribution (DC1) and a spatially correlated Gaussian random field (DC2). Two nonlinear mixture models were used: Bilinear Mixing Model (BLMM) and Post-Nonlinear Mixing Model (PNMM).
  • Real Data: A subscene of the Urban Hyperspectral Image (L=162 bands) was used, known to contain 3 endmembers: Asphalt, Vegetation, and Ground.

The authors also compare their method against several existing techniques, including FCLS, K-Hype, CDA-NL, and two other AEC-based methods (MF-AEC and NF-AEC).

Key Insights and Ideas

  • The Encoder as the Mixing Model's Inverse: The most significant insight is the explicit modeling of the encoder as the inverse of the decoder. This principled approach leverages prior knowledge about the unmixing problem to improve robustness and accuracy.
  • Structured Encoder for Robustness: By imposing a structure on the encoder derived from the pseudoinverse of the endmember matrix, the authors introduce valuable prior information. This constraint helps prevent the encoder from learning arbitrary mappings, leading to more physically meaningful and robust abundance estimates.
  • Balancing Linearity and Nonlinearity: The introduction of the parameter $\alpha$ allows for a principled way to balance the contributions of the linear and nonlinear components within the encoder, providing flexibility in adapting to different levels of nonlinearity.
  • Model-Based AECs for Nonlinearity: The paper demonstrates that by incorporating model-based constraints into AECs, significant improvements can be achieved in nonlinear hyperspectral unmixing, a task where model-free approaches can sometimes struggle with robustness.

Key Results

  • Improved Accuracy: The proposed MAC-U method consistently outperformed competing algorithms on both synthetic datasets (DC1 and DC2) across BLMM and PNMM models, achieving lower Root Mean Squared Errors for abundance maps (RMSEA).
  • Competitive Real-World Performance: On the Urban scene dataset, MAC-U produced well-defined and coherent abundance maps, particularly for Asphalt and Ground endmembers, outperforming the model-free MF-AEC and showing comparable or better results than other methods.
  • Reconstruction Quality: While RMSEY (image reconstruction error) for MAC-U was sometimes slightly larger than K-Hype, the authors note that RMSEY is not the primary metric for evaluating unmixing performance.
  • Computational Cost: The average execution times for MAC-U were higher than traditional methods like K-Hype and CDA-NL but comparable to other AEC-based strategies, which is a common trade-off for deep learning approaches.

Limitations and Future Work

  • Computational Cost: The paper acknowledges that the average execution times of MAC-U are significantly larger than traditional methods, suggesting that further optimization might be needed for real-time applications.
  • Parameter Tuning: While grid search was used for hyperparameter selection, the sensitivity of the model to these parameters and more efficient tuning strategies could be areas for future exploration.
  • Generalization to More Complex Models: The current work focuses on a specific form of nonlinear fluctuation. Future work could explore extending this model-based AEC approach to other, more complex nonlinear mixing models.
  • Endmember Variability: While the paper mentions spectral variability in the abstract and introduction, the proposed MAC-U architecture's explicit handling of endmember variability could be further investigated and enhanced.

Summary updated

Abstract

Autoencoder (AEC) networks have recently emerged as a promising approach to perform unsupervised hyperspectral unmixing (HU) by associating the latent representations with the abundances, the decoder with the mixing model and the encoder with its inverse. AECs are especially appealing for nonlinear HU since they lead to unsupervised and model-free algorithms. However, existing approaches fail to explore the fact that the encoder should invert the mixing process, which might reduce their robustness. In this paper, we propose a model-based AEC for nonlinear HU by considering the mixing model a nonlinear fluctuation over a linear mixture. Differently from previous works, we show that this restriction naturally imposes a particular structure to both the encoder and to the decoder networks. This introduces prior information in the AEC without reducing the flexibility of the mixing model. Simulations with synthetic and real data indicate that the proposed strategy improves nonlinear HU.
BibTeX
@article{li2021model,
  author = {Haoqing Li and Ricardo Augusto Borsoi and Tales Imbiriba and Pau Closas and José Carlos Moreira Bermudez and Deniz Erdogmus},
  title = {{Model-Based Deep Autoencoder Networks for Nonlinear Hyperspectral Unmixing}},
  journal = {IEEE Geosci. Remote. Sens. Lett.},
  year = {2021},
  doi = {10.1109/lgrs.2021.3075138},
}
Figure