A Data Dependent Multiscale Model for Hyperspectral Unmixing With Spectral Variability

Ricardo Augusto Borsoi, Tales Imbiriba, José Carlos Moreira Bermudez

TL;DR

This paper introduces MUA-SV, a novel multiscale algorithm for hyperspectral unmixing that effectively handles spectral variability—the real-world phenomenon where a material's "fingerprint" changes across an image. By using a data-dependent superpixel transform to separate the image into coarse and fine scales, the authors create a fast, parallelizable optimization process that outperforms state-of-the-art methods in both accuracy and speed.

What Problem Does This Paper Address?

In hyperspectral imaging, each pixel typically contains a mixture of materials (e.g., soil, grass, water). Hyperspectral Unmixing (HU) is the task of identifying these pure materials (endmembers) and their proportions (abundances) in every pixel.

The classic Linear Mixing Model (LMM) assumes that a material's spectral signature is constant throughout the scene. In reality, factors like changing illumination, topography, and atmospheric effects cause spectral variability. While newer models like the Extended Linear Mixing Model (ELMM) account for this, they often result in "ill-posed" math problems that are computationally grueling to solve, especially when trying to ensure that neighboring pixels have similar material proportions (spatial regularization).

Methods

The authors propose the Multiscale Unmixing Algorithm Accounting for Spectral Variability (MUA-SV). Its core innovation is a multiscale spatial mixture model:

  1. Superpixel Decomposition: The algorithm uses the SLIC (Simple Linear Iterative Clustering) method to group similar pixels into "superpixels." This creates a data-dependent transformation that respects the natural boundaries in the image.
  2. Multiscale Representation: The image is decomposed into two scales: a coarse scale (averages within superpixels) and a detail scale (pixel-level variations).
  3. Separable Optimization: By making a few reasonable physical assumptions—namely that residuals are uncorrelated across scales and that endmember signatures vary smoothly in space—the authors decouple the complex interdependencies between pixels.
  4. Fast Solver: This decoupling allows the abundance estimation to be solved independently for each pixel in both domains. This turns a massive, global optimization problem into many small, parallelizable ones.

Datasets

The researchers evaluated MUA-SV against three synthetic and two real-world datasets:

  • Synthetic (DC1, DC2, DC3): These were designed to test the algorithm against different types of variability, including a realistic simulation of illumination changes based on the Hapke model and a hilly terrain.
  • Houston Data Set: A 152×108 pixel subset of an urban hyperspectral image.
  • Cuprite Data Set: A 250×191 pixel subregion of the well-known mineralogical site in Nevada.

Key Insights and Ideas

  • Breaking the Bottleneck: Standard spatial regularization (like Total Variation) usually "ties" all pixels together in one giant math problem. MUA-SV uses the multiscale transform to achieve similar smoothing effects while keeping the problem "separable" (solvable pixel-by-pixel).
  • Physical Motivation: The model specifically targets illumination-induced variability, which is one of the most common and significant "non-ideal" effects in remote sensing.
  • Efficiency via Structure: The "aha moment" is realizing that because spectral variability is usually spatially smooth, we can solve for it primarily at a coarse scale and then refine it, rather than struggling with high-dimensional matrices at every single pixel from the start.

Key Results

MUA-SV demonstrated a clear "win-win" in performance:

  • Accuracy: It consistently achieved lower Mean Squared Error (MSE) for abundances compared to ELMM and PLMM across all noise levels (20dB to 40dB).
  • Speed: On the large Cuprite dataset, MUA-SV was approximately 5.5 times faster than ELMM and a staggering 80 times faster than PLMM.
  • Visual Quality: Qualitative results on the Houston and Cuprite images showed that MUA-SV produced cleaner, more homogeneous abundance maps that better delineated different materials without blurring sharp edges.

Limitations and Future Work

  • Smoothness Assumption: The algorithm relies on the assumption that endmember signatures don't change abruptly between adjacent pixels. While generally true, this might fail in scenes with extremely heterogeneous, small-scale features.
  • Superpixel Sensitivity: The performance is somewhat dependent on the initial superpixel segmentation. If the superpixels are too large, they might group semantically different materials, hindering accuracy.
  • Future Directions: The authors suggest that this multiscale framework could be extended to even more complex non-linear mixing models or applied to other types of hyperspectral data processing beyond unmixing.

Summary updated

Abstract

Spectral variability in hyperspectral images can result from factors including environmental, illumination, atmospheric and temporal changes. Its occurrence may lead to the propagation of significant estimation errors in the unmixing process. To address this issue, extended linear mixing models have been proposed which lead to large scale nonsmooth ill-posed inverse problems. Furthermore, the regularization strategies used to obtain meaningful results have introduced interdependencies among abundance solutions that further increase the complexity of the resulting optimization problem. In this paper we present a novel data dependent multiscale model for hyperspectral unmixing accounting for spectral variability. The new method incorporates spatial contextual information to the abundances in extended linear mixing models by using a multiscale transform based on superpixels. The proposed method results in a fast algorithm that solves the abundance estimation problem only once in each scale during each iteration. Simulation results using synthetic and real images compare the performances, both in accuracy and execution time, of the proposed algorithm and other state-of-the-art solutions.
BibTeX
@article{borsoi2020data,
  author = {Ricardo Augusto Borsoi and Tales Imbiriba and José Carlos Moreira Bermudez},
  title = {{A Data Dependent Multiscale Model for Hyperspectral Unmixing With Spectral Variability}},
  journal = {IEEE Trans. Image Process.},
  year = {2020},
  url = {https://openreview.net/forum?id=z-nAeFhydQ},
}

Loading related content...

Figure