Nonlinear Unmixing of Hyperspectral Images: Models and Algorithms.

Nicolas Dobigeon, Jean-Yves Tourneret, Cédric Richard, José C. M. Bermudez, Steve McLaughlin, Alfred O. Hero IiiIEEE Signal Process. Mag 2014 · OpenReview

TL;DR

This paper provides a comprehensive overview of recent advancements in nonlinear unmixing of hyperspectral images. It highlights the limitations of the traditional linear mixing model and explores various nonlinear models and algorithms that better capture complex interactions within hyperspectral data, crucial for more accurate material identification.

What Problem Does This Paper Address?

Hyperspectral imaging captures detailed spectral information across numerous contiguous bands, making it invaluable for identifying and quantifying materials in a scene. A core task in analyzing this data is "spectral unmixing," which aims to determine the constituent materials (endmembers) and their proportions (abundances) within each pixel. The vast majority of existing unmixing techniques rely on the Linear Mixing Model (LMM), which assumes that the spectrum of a mixed pixel is a simple linear combination of the spectra of pure materials. However, this assumption breaks down in many real-world scenarios, such as when light undergoes multiple scattering events or when materials are intimately mixed at a microscopic level. These nonlinear interactions lead to spectral signatures that cannot be accurately represented by the LMM, necessitating the development and study of nonlinear unmixing models and algorithms.

Methods

The paper categorizes nonlinear mixing models into two main families:

  • Intimate Mixtures: These models address scenarios where interactions occur at a microscopic level, such as with granular materials. They often draw from radiative transfer theory and models like Hapke's.
  • Bilinear Models: These models account for macroscopic interactions where light scattered by one material interacts with another before reaching the sensor. Several variations are discussed, including:
  • Nascimento Model (NM): Interpreted as an LMM with additional "virtual" endmembers.
  • Fan Model (FM): Relates nonlinearity coefficients directly to abundance coefficients.
  • Generalized Bilinear Model (GBM): A flexible model that can generalize LMM and FM.
  • Linear-Quadratic Mixing Model (LQM): Includes quadratic terms for interactions.
  • Polynomial Post-Nonlinear Model (PPNM): A general parametric model that can encompass many bilinear models.

The paper also reviews various unmixing algorithms, broadly classified into:

  • Model-based parametric algorithms: These algorithms explicitly rely on a defined nonlinear mixing model. They can be supervised (endmembers known) or unsupervised (endmembers unknown). Techniques include neural networks, optimization methods (e.g., constrained least squares, gradient descent), and Bayesian approaches.
  • Model-free algorithms: These approaches do not assume a specific nonlinear mixing model and often leverage machine learning techniques like kernel-based methods (e.g., reproducing kernel Hilbert spaces) and Gaussian processes latent variable models (GPLVMs).

Furthermore, the paper discusses methods for detecting nonlinear mixtures as a pre-processing step, allowing linear unmixing for pixels that conform to the LMM and reserving more complex nonlinear methods for identified nonlinear pixels.

Datasets

The paper uses synthetic datasets generated according to various mixing models (LMM, GBM, FM, PPNM) for evaluating the performance of different unmixing algorithms. These synthetic images are of size 50x50 pixels and are corrupted with additive Gaussian noise. The paper also refers to real hyperspectral data, such as the Madonna dataset acquired over Villelongue, France, for illustrative purposes.

Key Insights and Ideas

  • The LMM is often insufficient: Real-world hyperspectral data frequently exhibits nonlinear mixing effects, making the LMM a poor approximation and leading to inaccurate results.
  • Nonlinear models offer richer representations: Intimate mixtures and bilinear models provide more physically plausible ways to describe complex interactions, leading to potentially more accurate unmixing.
  • Model-based vs. Model-free: Model-based approaches offer interpretability but require accurate model specification, while model-free methods offer flexibility but can be computationally intensive and less interpretable.
  • Detection is crucial: Identifying pixels that are truly nonlinear before applying complex algorithms can significantly improve efficiency and accuracy.
  • Endmember extraction remains challenging: While models are advancing, robustly extracting endmembers in the presence of strong nonlinearities is still an active research area.

Key Results

The paper presents a table (Table I) comparing the abundance estimation performance (measured by RNMSE) of various unmixing strategies across different mixing models and scenarios (with/without pure pixels). Key findings include:

  • Prior knowledge of the mixing model is advantageous: Using an algorithm that matches the underlying mixing model significantly improves abundance estimation accuracy.
  • PPNM shows flexibility: The Polynomial Post-Nonlinear Model (PPNM) demonstrates good performance across various mixing scenarios, suggesting its adaptability.
  • Model-free approaches can be competitive: The K-HYPE algorithm, a model-free approach, shows promising results, especially when prior knowledge of the mixing model is absent.
  • Nonlinear detection is effective: Figure 4 illustrates the successful detection of linear versus nonlinear pixels using a PPNM-based detector, showing distinct clusters for different mixing models.

Limitations and Future Work

  • Computational Complexity: Nonlinear unmixing methods generally incur higher computational costs compared to LMM-based approaches, which can be a bottleneck for large hyperspectral images.
  • Heterogeneity of the Medium: Current models often assume homogeneous regions. Future work should focus on developing unmixing models that can handle heterogeneous regions, where linear, weakly nonlinear, and strongly nonlinear pixels coexist.
  • Spatial Information: Exploiting spatial information inherent in hyperspectral images, beyond pixel-by-pixel analysis, is a significant challenge for developing more robust unsupervised algorithms.
  • Integration of Physical Models: Better integration of algorithmic approaches with physical models (e.g., radiative transfer effects like scattering, dispersion) is needed to improve nonlinear unmixing performance.
  • Unknown Number of Endmembers: The presence of nonlinearity is often related to the number of endmembers, which is typically unknown. Developing methods that can jointly estimate the number of endmembers and handle nonlinearities is an important future direction.
  • Manifold Learning: Advances in manifold learning and dimensionality estimation are promising avenues for addressing the nonlinear unmixing problem.

Summary updated

Abstract

When considering the problem of unmixing hyperspectral images, most of the literature in the geoscience and image processing areas relies on the widely used linear mixing model (LMM). However, the LMM may be not valid, and other nonlinear models need to be considered, for instance, when there are multiscattering effects or intimate interactions. Consequently, over the last few years, several significant contributions have been proposed to overcome the limitations inherent in the LMM. In this article, we present an overview of recent advances in nonlinear unmixing modeling.
BibTeX
@inproceedings{dobigeon2014nonlinear,
  author = {Nicolas Dobigeon and Jean-Yves Tourneret and Cédric Richard and José C. M. Bermudez and Steve McLaughlin and Alfred O. Hero III},
  title = {{Nonlinear Unmixing of Hyperspectral Images: Models and Algorithms.}},
  booktitle = {IEEE Signal Process. Mag},
  year = {2014},
  url = {https://openreview.net/forum?id=Yby62v4xqs9},
}
Figure