When a satellite or airborne sensor captures an image of the Earth's surface, a single pixel rarely contains just one pure material. Instead, it records a mixture of signatures—soil, water, and various types of vegetation all blended together. The classical approach to decoding this signal is the Linear Mixture Model (LMM). The LMM operates on a simple "checkerboard" assumption: it posits that the ground is perfectly flat and divided into distinct patches of pure materials (endmembers). Under this assumption, a photon strikes exactly one material and reflects directly back to the sensor, making the observed pixel spectrum a simple, weighted additive combination of the constituent material signatures.
However, the real world is rarely flat or neatly partitioned. When light interacts with complex three-dimensional structures—such as dense forest canopies, deep urban canyons, or microscopic particulate mixtures—the linear assumption breaks down entirely.
The Physics of Multi-Path Interference
To understand why the linear model fails, we must look at how photons actually travel through a scene. In an environment with geometric complexity, a photon rarely undergoes a single reflection. Instead, it experiences multiple scattering.
Imagine a photon entering a dense forest canopy. It might first strike a tree leaf, absorb certain wavelengths, and reflect the rest downward onto the soil. From the soil, it bounces back upward, hitting the underside of another branch before finally escaping the canopy and traveling toward the remote sensor.
Linear Mixing (Checkerboard) Non-Linear Mixing (Multi-Path)
Sensor [↑] [↑] Sensor [↑]
│ │ │
┌────┴─────┴────┐ ┌─────┴──────────┐
│ Grass │ Soil │ │ Leaf ──┐ │
└───────┴───────┘ │ ▼ │
│ Soil ───┐ │
│ ▼ │
│ Leaf│
└────────────────┘
Because the light has interacted with multiple surfaces sequentially, the resulting spectral signature is no longer a simple weighted average. Instead, it contains cross-products and higher-order polynomial combinations of the individual material reflectances. As noted by Feiyun Zhu et al. in Structured Sparse Method for Hyperspectral Unmixing, ignoring these complex physical interactions introduces significant errors when estimating the true proportions (abundances) of materials in topographically or structurally complex environments.
This phenomenon manifests in two primary ways:
- Macroscopic Non-Linearity: Caused by large-scale 3D geometry, such as shadows, uneven terrain, and structural cavities (e.g., urban canyons or tree rows).
- Microscopic (Intimate) Non-Linearity: Occurs when different materials are intimately blended at a particulate level, such as mineral powders, liquids, or layered paint pigments.
In Unmixing Hyperspectral Data, Lucas C. Parra et al. emphasize that intimate mixtures are inherently non-linear because light scatters repeatedly between neighboring particles before escaping the medium. This particulate-level scattering is also a major hurdle in cultural heritage applications. For example, Shuqiang Lyu et al. demonstrate in Research on the Hyperspectral-based Unmixing Model of Composite Pigments on Mural Surfaces that standard linear or simple logarithmic transformations fail to isolate individual pigments on ancient artwork because the physical scattering between tightly packed pigment layers violates basic additive assumptions.
Bridging Physics and Computational Models
Accounting for multiple scattering requires moving beyond simple linear algebra to models that can ingest or approximate non-linear physical dynamics.
One rigorous path relies on analytical radiative transfer formulations, such as the Hapke model, which explicitly describes how light propagates through particulate media. However, these physical models are highly complex and computationally expensive to invert directly. To bridge this gap, researchers often use mathematical approximations that retain physical meaning.
In Spectral Unmixing: A Derivation of the Extended Linear Mixing Model From the Hapke Model, Lucas Drumetz et al. show that the Extended Linear Mixing Model (ELMM)—which uses scaling factors to accommodate variations in illumination and topography—can be mathematically derived as a first-order Taylor expansion of the complex Hapke model. While this provides a solid physical justification for using scaling factors, the authors demonstrate that such linear approximations still struggle with high-albedo (highly reflective) materials, where higher-order multiple-scattering effects dominate and break the first-order assumption.
To capture these higher-order interactions without relying on explicit physical parameters, researchers have turned to non-linear regression techniques. For instance, Bo Wu et al. in Unmixing Hyperspectral Imagery Based on Support Vector Nonlinear Approximating Regression utilize Support Vector Regression (SVR) to map the complex, non-linear relationships introduced by shadows and uneven terrain.
While non-linear models are highly accurate, mapping every pixel into a high-dimensional feature space introduces a massive computational burden. To solve this "big data" bottleneck, Tales Imbiriba et al. in Band selection in RKHS for fast nonlinear unmixing of hyperspectral images propose performing intelligent band selection within a Reproducing Kernel Hilbert Space (RKHS). By isolating the most informative spectral bands within a kernelized framework, they capture non-linear multi-path interactions while keeping the computational cost manageable for large-scale imaging tasks.
Learning Disentangled Physical Representations
The latest evolution in non-linear spectral unmixing leverages deep learning. Instead of hand-crafting non-linear polynomial terms or kernel functions, neural networks can learn the complex geometry of light transport directly from data. However, standard "black-box" neural networks often yield uninterpretable results that violate physical constraints, such as conservation of energy (abundances must be non-negative and sum to one).
To address this, modern architectures embed physical constraints directly into the network design using disentanglement learning. The goal is to force the network to separate the mixing process into two distinct latent components: the true, linear material proportions (abundances) and the non-linear environmental distortions (spectral variability and multi-path interference).
┌─► Linear Abundances (True Proportions)
│
Mixed Pixel ──► ID-Net │
│
└─► Non-Linear Interactions (Scattering/Shadows)
In Learning Interpretable Deep Disentangled Neural Networks for Hyperspectral Unmixing, Ricardo Augusto Borsoi et al. introduce ID-Net, a deep learning framework that utilizes a physically motivated architecture to isolate material proportions from non-linear scattering effects. By enforcing this structural separation, the network remains fully interpretable to researchers.
The authors expand on this concept in A Deep Disentangled Approach for Interpretable Hyperspectral Unmixing with PhI-Net, which uses variational inference to model the non-linear multi-path interference as a distinct, latent environmental factor. By treating non-linearity as an explicit component to be isolated rather than noise to be ignored, these models successfully reconstruct the underlying physical reality of a scene, proving that deep networks can be both highly non-linear and strictly bound by the laws of physics.
Key Foundational Work
The Estimation of Vegetation Variables Based on High Resolution Spectra (1997) — contributed 1 concept elements to this direction.
Learning a Color Algorithm from Examples (1987) — contributed 1 concept elements to this direction.