Physics-Guided Deep Learning for Image Restoration

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2415 papers423 conceptsfrozen 2026-10-04

Deep learning has revolutionized image restoration, but "black-box" neural networks often struggle with a fundamental flaw: they are prone to hallucinations. When a standard convolutional network attempts to remove haze, blur, or noise, it relies on patterns learned from training data rather than the actual laws of optics. This can lead to reconstructions that look visually pleasing but are physically impossible.

To solve this, a growing body of research integrates physical principles—such as light scattering, diffraction, and signal acquisition geometry—directly into neural architectures. By anchoring deep learning in the "forward models" of physics, researchers are creating systems that are more accurate, interpretable, and robust.

The Problem with Black-Box Restoration

In traditional deep learning, a model is trained to map a degraded input (like a blurry image) to a clean output. However, as noted by Gabriel Della Maggiora and Artur Yakimovich in their work on microscopy datasets, these models often generate structures that a microscope could not have physically recorded.

Without physical grounding, models suffer from three main issues:

  1. Inconsistency: The restored components (like reflectance and shading) may not mathematically reconstruct the original image, a problem highlighted by Anil S. Baslamisli and Theo Gevers in Intrinsic Image Decomposition.
  2. Generalization Failure: A model trained on one type of haze or blur often fails when the environment changes because it hasn't learned the underlying mechanism of light propagation.
  3. Artifacts: Unconstrained models may "cheat" by shifting pixels to compensate for misalignment rather than actually inverting the blur process, as observed by Dongwei Ren and Yu Li in Single Image Defocus Deblurring.

Embedding the Forward Model

The core intuition of physics-guided learning is to treat the neural network not as a replacement for physics, but as a way to parameterize it.

In image dehazing, for instance, the physical process is governed by the atmospheric scattering model, which describes how light is attenuated and scattered by particles. Jin Xie and Xuelong Li developed HRGAN, which jointly estimates atmospheric light and transmission maps. By forcing the network to predict these physical parameters, the resulting image is guaranteed to adhere to the laws of light scattering.

Similarly, Hongyuan Zhu and Xi Peng introduced DehazeGAN, which uses differentiable programming to embed the scattering model into the network's loss function. This ensures that the model's internal logic remains interpretable: it isn't just "guessing" pixels; it is solving for the physical variables that created the haze.

Hybrid Architectures and Learnable Filters

Another approach involves "unrolling" classical physics-based algorithms into neural network layers. Instead of discarding decades of signal processing theory, researchers replace rigid components of these algorithms with learnable modules.

In Microscopy Image Restoration with Deep Wiener-Kolmogorov Filters, Filippos Kokkinos and Stamatios Lefkimmiatis embed deep "learnable regularizers" into the classical Wiener filter. This combines the physical grounding of the Point Spread Function (PSF) with the content-adaptive power of neural networks.

This hybrid strategy is also seen in medical imaging. Xi Tan and Jing Wang developed DeepFBP, which improves CT reconstruction by replacing the rigid interpolation steps of Filtered Back Projection with neural modules. By maintaining the geometry of the signal acquisition process, the model preserves structural integrity better than generic denoisers.

Constraining the Solution Space

Perhaps the most powerful application of this idea is in "seeing the unseen." When light is highly scrambled—such as when looking through a wall or deep into biological tissue—standard networks often produce noise.

Yiwei Ouyang and colleagues demonstrated this with WavePCNet, which detects objects hidden behind walls. By constraining the network with the laws of Fresnel diffraction, the model can differentiate between valid signal patterns and stochastic noise. The physics acts as a filter, narrowing the "solution space" to only those images that could have physically generated the observed light patterns.

This principle extends to 3D brain imaging. In XLFM-Former, Feng He and Jun Yu use a physics-based rendering loss to ensure that reconstructed neurons are consistent with the optics of light-field microscopy.

Why It Works: The Inductive Bias of Reality

The success of these methods stems from a better inductive bias. While a standard neural network can theoretically learn any mapping, a physics-guided network is "biased" toward mappings that exist in the real world.

By incorporating parameters like the refractive index structure constant for turbulence estimation or medium transmission maps for underwater restoration, these models require less data to achieve higher accuracy. They don't have to learn the laws of physics from scratch; they are given the laws as a blueprint and use data to fill in the specific details of the scene.

Papers in this version

Showing 60 of 415 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 227 papers
  2. Version 1 · 2026-10-04 · 415 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_physics_agnostic_modeling_3a757693,
  title = {physics-agnostic modeling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/physics-agnostic-modeling/rdr_3a757693bed645cb969b890ee1986907},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_physics_guided_deep_learning_for_image_r_3a757693_v1,
  title = {Physics-Guided Deep Learning for Image Restoration},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/physics-guided-deep-learning-for-image-restoration/rdr_3a757693bed645cb969b890ee1986907/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}