In traditional hyperspectral imaging, we often assume that a material—say, a specific type of soil or a species of oak tree—can be represented by a single, "pure" spectral signature called an endmember. If a pixel contains a mix of materials, we treat it as a simple linear combination of these fixed templates.
However, the real world is rarely that consistent. A leaf’s reflectance changes with its moisture content; a roof looks different under a cloud than in direct sunlight. This phenomenon, known as spectral variability, makes traditional hyperspectral unmixing (HU) brittle. If your "template" for grass doesn't perfectly match the grass in the scene, the model fails.
To solve this, researchers are moving away from single-vector templates toward Endmember Bundles and Spectral Dictionaries. Instead of representing a material as a point, they represent it as a manifold or a collection of possibilities.
From Points to Bundles
The core intuition is that a material class is better described as a "bundle" of potential signatures rather than a single average. Gokul Bhusal, Cristina García–Cardona, and Ekaterina Merkurjev argue in A General Framework for Group Sparsity in Hyperspectral Unmixing Using Endmember Bundles that this approach allows the system to adapt to pixel-by-pixel changes in lighting and moisture.
By treating each material as a group of signatures, the unmixing problem becomes a selection task: for this specific pixel, which version of "oak" best explains the data? This is often solved using group sparsity. The model is encouraged to pick only a few material classes (sparsity) but can choose any combination of signatures from within those classes' bundles to achieve the best fit.
Capturing Intra-class Variability
This shift is equally vital for target detection. When looking for a specific material in a cluttered environment, a single signature is often too restrictive. Susan Meerdink, Alina Zare, and their colleagues developed algorithms like MTMI-ACE in Multi-Target Multiple Instance Learning for Hyperspectral Target Detection to learn these dictionaries automatically.
By using Multiple Instance Learning (MIL), they can train detectors on "bags" of pixels. Even if the training labels are imprecise or the pixels are "mixed" with background noise, the system learns to identify the recurring spectral signatures that define the target class across different environments.
Convexity vs. Sparsity
How exactly should we select from these libraries? While many methods rely on sparsity (picking one or two "best" matches from a library), recent work suggests that convex combinations might be more robust.
In Fast Semi-supervised Unmixing using Non-convex Optimization, Alexandre Zouaoui, Behnood Rasti, and their team demonstrate that enforcing convexity constraints—allowing the model to "blend" library spectra—outperforms simple sparsity. This allows the model to approximate a spectrum that might not exist exactly in the library but is a physically plausible variation of the material caused by atmospheric effects or sensor noise.
Multi-scale Integration
The challenge of using large libraries is the computational cost and the risk of "overfitting" a pixel to the wrong material. To combat this, Luciano Carvalho Ayres, Ricardo Augusto Borsoi, and J.C.M. Bermudez proposed a multi-scale framework in Uma Abordagem Multi-escala para Separação Espectral Esparsa e Estruturada em Imagens Hiperespectrais.
By looking at the image at multiple scales, the system can use spatial context to inform the spectral selection. If neighboring pixels are all identified as "water," the model is more likely to select a "water" signature from its bundle for the current pixel, even if a "dark soil" signature provides a slightly better mathematical fit in isolation.
Why It Works
The transition from single endmembers to bundles works because it acknowledges the physical uncertainty of remote sensing. By modeling materials as distributions or sets rather than fixed points, these frameworks:
- Absorb Environmental Noise: Variations in shadow and moisture are treated as internal diversity of the material rather than external error.
- Improve Robustness: They reduce the "mismatch" between laboratory-grade spectral libraries and messy, real-world satellite data.
- Enable Flexibility: They allow for "mixed" pixels to be decomposed more accurately by finding the specific version of a material present in that exact location.