Spectral Variability in�Hyperspectral Data�Unmixing: A Comprehensive Review

Ricardo Borsoi, Tales Imbiriba, Jose Carlos Bermudez, Cedric Richard, Jocelyn Chanussot, Lucas Drumetz, Jean-Yves Tourneret, Alina Zare, Christian JuttenIEEE Geoscience and Remote Sensing Magazine 2021 · OpenReview

TL;DR

This comprehensive review tackles the pervasive issue of spectral variability in hyperspectral unmixing (SU), where material signatures change due to environmental factors. The authors propose a novel taxonomy to guide practitioners through existing and emerging SU methods designed to handle this variability, offering a structured overview of techniques from library-based approaches to blind methods.

What Problem Does This Paper Address?

Hyperspectral imaging captures detailed spectral information, enabling precise material identification. However, the spectral signatures of materials (endmembers or EMs) are rarely constant. Factors like atmospheric conditions, illumination changes, and intrinsic material variations cause these signatures to shift across an image. Traditional SU algorithms, which assume fixed EM signatures, struggle with this variability, leading to inaccurate abundance estimates and compromised results. This paper addresses the critical need for SU methods that can robustly handle this spectral variability.

Methods

The paper provides a comprehensive review of SU methods designed to address spectral variability. It introduces a novel taxonomy, organized as a decision tree, to help practitioners choose appropriate algorithms based on factors like the availability of spectral libraries, the required level of user supervision, and computational cost. The review categorizes methods into several families:

  • Library-based methods: These methods utilize pre-existing spectral libraries. They are further divided into:
  • MESMA and variants: Iteratively search for optimal EM combinations from libraries.
  • Sparse SU: Formulates SU as a sparse regression problem, selecting a minimal set of signatures.
  • Machine Learning: Employs supervised learning to map mixed pixels to abundances and EMs.
  • Spectral Transformations: Pre-processing steps to transform spectral data to mitigate variability.
  • Blind methods: These methods do not require prior spectral libraries. They include:
  • Local Unmixing: Performs SU on small image regions where EMs are assumed constant.
  • Parametric Models: Represent EM spectra as functions of a few parameters, allowing for controlled variability.
  • EM-model-free methods: Employ strategies to improve robustness without explicit EM models.
  • Bayesian Methods: Model EMs as random variables following statistical distributions.

The paper also discusses techniques for spectral library construction (image-based extraction, physics-based synthesis, spatial interpolation) and pruning (library reduction, endmember selection, same-class pruning).

Datasets

The paper does not explicitly list specific datasets used for training or evaluation in the review itself. However, it references numerous studies that utilize various hyperspectral datasets, including those from airborne sensors like AVIRIS and spaceborne sensors like Hyperion, as well as laboratory spectral libraries (e.g., USGS library). For their illustrative simulation, they generated synthetic hyperspectral data containing vegetation, dirt, and water, with spatially correlated abundances and endmember variability simulated using models like PROSPECT-D and simplified Hapke and atmospheric models.

Key Insights and Ideas

  • Spectral variability is ubiquitous and impactful: The paper strongly emphasizes that spectral variability is not an edge case but a fundamental challenge in hyperspectral SU, significantly degrading the performance of traditional methods.
  • A practitioner-centric taxonomy is crucial: The proposed decision-tree taxonomy is a key contribution, offering a practical framework for navigating the complex landscape of SU methods based on real-world constraints like data availability and computational resources.
  • Library-based methods offer a trade-off: While library-based methods can yield good results, their performance is highly dependent on the quality and representativeness of the spectral library. The development of methods for library extraction and pruning is therefore critical.
  • Blind methods offer flexibility but pose challenges: Blind methods, which don't rely on prior libraries, are valuable when libraries are unavailable. However, they often require more user supervision for parameter tuning or can be computationally intensive.
  • Parametric and Bayesian models offer sophisticated variability modeling: These approaches provide more physically grounded or statistically robust ways to represent and handle EM variability, though they can introduce their own complexities in terms of parameter estimation and computational cost.

Key Results

The paper is a comprehensive review, not an experimental study presenting novel results. However, it synthesizes findings from numerous studies. The illustrative simulation demonstrates that methods accounting for spectral variability generally outperform traditional methods like FCLS. Specifically, library-based methods (MESMA, fractional sparse SU) showed good performance when the extracted library was representative. Parametric models (ELMM, DeepGUn) also offered significant improvements. Bayesian methods, particularly BCM, showed considerable performance gains, especially in unsupervised settings. The simulation also highlighted significant differences in computational cost, with library-based methods being very fast, parametric models intermediate, and Bayesian methods the slowest.

Limitations and Future Work

The authors acknowledge several limitations and suggest future research directions:

  • User supervision and parameter tuning: Many advanced SU methods still require significant user input and careful parameter tuning, hindering their out-of-the-box applicability. Future work should focus on developing informed methodologies for parameter adjustment, potentially leveraging metadata.
  • Robustness to library quality: The performance of library-based methods is highly sensitive to the quality and representativeness of spectral libraries. Improving robustness in this regard is crucial.
  • Nonlinear mixing models: The paper notes that while linear unmixing with spectral variability can partially address nonlinear mixtures, the relationship between these models and the ability to distinguish between spectral variability, nonlinear interactions, and abundance variations warrants further investigation.
  • Lack of reliable ground truth data: Evaluating SU methods is challenging due to the scarcity of reliable ground truth data for real hyperspectral images, especially when spectral variability is considered. The development of a larger, publicly available dataset and standardized protocols for generating realistic synthetic data is needed.
  • Theoretical insights: While traditional SU has a clear theoretical foundation, a deeper understanding of the conditions under which SU problems with endmember variability can be solved exactly is still lacking, except for specific cases like illumination variability.
  • Computational complexity: Many algorithms remain computationally expensive, especially for large images. Developing faster alternatives is essential for practical applications.
  • Bridging theory and practice: Combining theoretical insights with practical application knowledge is seen as a promising avenue for developing more effective models for spectral variability.

Summary updated

Abstract

The spectral signatures of the materials contained in hyperspectral images, also called <i>endmembers</i> (<i>EMs</i>), can be significantly affected by variations in atmospheric, illumination, and environmental conditions that typically occur within an image. Traditional spectral unmixing (SU) algorithms neglect the spectral variability of the EMs, which propagates significant modeling errors throughout the whole unmixing process and compromises the quality of the results. Therefore, serious efforts have been dedicated to mitigating the effects of spectral variability in SU. This resulted in the development of algorithms that incorporate different strategies to enable the EMs to vary within a hyperspectral image, using, for instance, sets of spectral signatures known a priori as well as Bayesian, parametric, and local EM models.
BibTeX
@article{borsoi2021spectral,
  author = {Ricardo Borsoi and Tales Imbiriba and Jose Carlos Bermudez and Cedric Richard and Jocelyn Chanussot and Lucas Drumetz and Jean-Yves Tourneret and Alina Zare and Christian Jutten},
  title = {{Spectral Variability in�Hyperspectral Data�Unmixing: A Comprehensive Review}},
  journal = {IEEE Geoscience and Remote Sensing Magazine},
  year = {2021},
  url = {https://openreview.net/forum?id=oyMdCERpAE},
}
Figure