TL;DR
This paper introduces the Illumination Invariant Spectral Unmixing (IISU) model, which leverages LiDAR-derived Digital Surface Models (DSMs) to explicitly account for illumination variations and shadows in hyperspectral image unmixing. By physically modeling light interactions and using radiance data directly, IISU significantly outperforms existing methods, especially in challenging shaded regions.
What Problem Does This Paper Address?
Hyperspectral unmixing aims to identify the constituent materials (endmembers) within a pixel and their fractional abundances. A major challenge is spectral variability, where the same material can exhibit different spectral signatures due to varying illumination conditions, particularly shadows caused by topography. Existing unmixing models often struggle with this, either by oversimplifying the problem or by relying on reflectance data that has already been affected by atmospheric and illumination effects. The paper highlights that current methods fail to physically explain why spectral variability occurs and often perform poorly in shaded pixels, leading to inaccurate abundance estimates and shadow-compensated reflectance.
Methods
The core contribution is the Illumination Invariant Spectral Unmixing (IISU) model. Unlike previous approaches that typically work with reflectance data, IISU directly processes radiance data and incorporates physical parameters derived from a LiDAR-derived Digital Surface Model (DSM). These DSM-derived parameters include:
- Visibility (v): Whether a pixel is directly illuminated by the sun.
- Sky Factor (f): The proportion of diffuse skylight reaching the surface.
- Incident Angle (θ): The angle between the sun's direction and the surface normal.
IISU uses a simplified radiative transfer model to explain how these factors influence the observed radiance. It explicitly models direct sunlight, diffused skylight, and indirect illumination. The model is solved using a two-step optimization process:
- Preprocessing: Estimates sunlight (s¹) and skylight (s²) parameters using a block coordinate descent approach.
- Abundance Estimation: Estimates endmember abundances (a) and bilinear endmember coefficients (e) using a non-negative least squares approach.
The paper also provides a theoretical framework to explain the physical meaning of various existing unmixing models by relating them to IISU's more comprehensive physical model.
Datasets
The paper evaluates IISU using both simulated and real-world datasets:
- Simulated Datasets:
- SIM1: A simple, homogeneous dataset with three regions (trees, soil, building) designed to highlight failures in shaded pixels.
- SIM2: A more realistic dataset with seven endmember classes, generated using a real DSM.
- Real Datasets:
- RIM1: Hyperspectral imagery and DSM acquired over the University of Houston campus, featuring cloud shadows.
- RIM2: Hyperspectral imagery and DSM from the Data Fusion Contest 2018, focusing on urban areas with higher spatial resolution.
Key Insights and Ideas
- Physical Explanation of Spectral Variability: IISU moves beyond empirical modeling by physically explaining spectral variability as a consequence of illumination geometry (sun angle, visibility, sky factor) and indirect light.
- DSM as a Key Enabler: LiDAR-derived DSMs are crucial for providing the geometric information needed to model illumination and shadows accurately, which is a significant advancement over previous DSM-based methods that primarily used DSM for spatial regularization.
- Radiance-Based Unmixing: Directly unmixing radiance data, rather than reflectance, simplifies the process by avoiding the need for a separate, potentially error-prone atmospheric correction step.
- Unified Framework: IISU integrates atmospheric correction considerations (illumination modeling) directly into the unmixing framework, addressing a long-standing limitation.
- Improved Performance in Shadows: The explicit modeling of illumination and shadows leads to significantly better performance in shaded pixels, where traditional methods fail.
Key Results
- Quantitative Performance: IISU consistently achieved the lowest Root Mean Square Errors (RMSE) for both abundance estimation and shadow-compensated reflectance across all simulated and real datasets compared to FCLS, FCLS-s, SCLS, NLMM, and U-DSM. For example, in SIM1, IISU's RMSEa was 1.2175 × 10⁻⁴, significantly lower than the next best method (U-DSM at 7.8584 × 10⁻⁴).
- Qualitative Performance: Visualizations of abundance maps and shadow-compensated reflectance clearly show IISU's superior ability to handle shaded areas, producing more accurate and realistic results. Existing methods often misclassify shaded pixels or fail to recover their true spectral properties.
- Robustness: IISU demonstrated robustness in handling variable illuminations and shadows, outperforming methods that only partially account for spectral variability or use DSM for spatial regularization.
Limitations and Future Work
- Limitations:
- The paper acknowledges that the current model primarily considers linear and bilinear light interactions. More complex nonlinear interactions (e.g., in dense canopies) might require further model extensions.
- While IISU is less sensitive to atmospheric effects by working with radiance, it still assumes a simplified radiative transfer model.
- The preprocessing step for estimating sunlight and skylight parameters relies on manually identifying pixels in sunlit and shadow conditions.
- Future Work:
- Extending the model to incorporate more complex nonlinear mixing models (e.g., intimate mixtures, multilinear mixtures).
- Investigating the use of other constraints (e.g., sparsity, spatial regularization) to further improve abundance estimation accuracy.
- Exploring spectral variability caused by factors other than illumination, such as intrinsic material properties.
- The authors suggest that incorporating orthophoto-derived DSM could be of interest for future work.
Summary updated