Computer vision often assumes a "clear day" scenario where light travels unimpeded from an object to the sensor. In reality, the world is filled with scattering media: fog, rain, murky water, and even the paper of a sealed envelope. Traditional deep learning treats restoring these images as a "black-box" translation task—mapping a blurry input to a sharp output. However, a growing body of research suggests that the most robust way to "see through" degradation is to embed the laws of physics directly into the neural network's architecture.
The Limits of Black Box Restoration
Purely data-driven models, such as standard Generative Adversarial Networks (GANs), often struggle with real-world degradation because they lack an internal model of how light behaves. As Wenhan Yang and Robby T. Tan point out in their roadmap of image deraining, black-box models frequently over-smooth textures or fail to generalize when the "rain" in the real world looks different from the synthetic training data.
The core problem is ambiguity. A gray pixel in a foggy image could be a white car far away or a dark car nearby. Without understanding the relationship between depth and light attenuation, a network is essentially guessing.
Embedding the Forward Model
The breakthrough in this field involves integrating the Atmospheric Scattering Model (or similar physical "forward models") into the learning process. This model mathematically describes how an image is formed:
$$I(z) = J(z)t(z) + A(1 - t(z))$$
Where $I$ is the hazy image, $J$ is the clear scene, $t$ is the transmission map (how much light gets through), and $A$ is the global atmospheric light.
Instead of asking a network to jump straight from $I$ to $J$, researchers like Yanwei Pang and Jin Xie in BidNet use dual-branch architectures. One branch estimates the physical parameters ($t$ and $A$), while the other uses those parameters to reconstruct the image. This ensures the output is not just visually pleasing, but physically plausible.
Physical Priors as Structural Inductive Biases
Beyond simple equations, researchers are using physical "priors"—statistical properties of the physical world—to guide networks.
- Dark Channel Prior (DCP): In clear images, at least one color channel has very low intensity in most local patches. Wei-Ting Chen and colleagues in JSTASR integrate this as a differentiable layer, allowing the network to use this classical heuristic to detect snow and haze during end-to-end training.
- Transmission-Aware Attention: In DeHamer, the authors incorporate haze density directly into a Transformer's spatial awareness. By using a "3D Position Embedding" that accounts for how light fades over distance, the model learns to pay more attention to regions where the degradation is most severe.
- Polarization and Reflectance: To solve niche problems like "seeing through" wrinkled film, Jiaqi Tang uses polarized priors, leveraging the fact that reflected glare is polarized differently than the object behind it.
Unfolding Classical Algorithms
Another powerful technique is "algorithm unfolding." Classical restoration used iterative optimization (like Wiener filters) which were mathematically sound but slow and required manual tuning.
Weisheng Dong and Guangming Shi demonstrate this in their denoising prior driven network. They take a traditional iterative algorithm and "unfold" it into a neural network where each layer represents one iteration. Similarly, Filippos Kokkinos and Stamatios Lefkimmiatis use Deep Wiener-Kolmogorov Filters for microscopy. This hybrid approach keeps the physical "skeleton" of the classical filter while using deep learning to fill in the "muscles"—the complex, data-driven details that classical math can't easily capture.
Why It Works: Constraining the Search Space
The intuition for why this works is simple: constraint. The space of all possible images is infinite. When we train a black-box model, we are asking it to find a needle in a haystack.
By injecting physical laws—whether it's the Point Spread Function in non-line-of-sight imaging by Feihu Xu or Retinex theory in low-light enhancement by Yinglong Wang—we shrink the haystack. We tell the model: "The solution must not only look like a clear image but must also explain how the light from the original scene was scattered to create the blurry input."
This grounding prevents the model from falling into "local minima" where it creates artifacts that look good to a discriminator but violate the laws of optics. As demonstrated by Bingyao Huang and Haibin Ling in their work on reading sealed mail, this physical modeling is often the difference between seeing a blurry mess and recovering high-fidelity hidden information.