Early image deraining methods operated under a simple assumption: rain is an additive layer of linear streaks superimposed on a clean background. Mathematically, this was written as:
$$I = B + R$$
where $I$ is the observed rainy image, $B$ is the clean background, and $R$ represents the rain streaks. Under this paradigm, deraining was treated primarily as a signal separation or high-frequency filtering problem.
However, as noted by He Zhang et al. in their work on conditional GANs, this model breaks down completely in heavy downpours. In torrential rain, the atmosphere becomes saturated with a dense concentration of water droplets. Instead of distinct, isolated streaks, the accumulation of countless distant rain particles scatters light, creating a thick, fog-like "rain veil."
This atmospheric veiling effect drastically reduces global contrast, washes out colors, and obscures fine structural details. Simply subtracting streaks leaves behind a hazy, low-visibility image because the model fails to account for the volumetric scattering of light.
Deconstructing the Physics of Heavy Rain
To effectively restore images taken in heavy rain, researchers had to move beyond simple additive models and rethink the underlying physical degradation. Rather than treating rain as a single layer, modern frameworks decompose the degradation into two distinct but physically entangled phenomena: localized rain streaks and global atmospheric accumulation.
[Clean Background] ---> [Atmospheric Scattering / Haze] ---> [Rain Streak Occlusion] ---> [Observed Image]
Ruoteng Li et al. formalized this by integrating a physics-based model that treats heavy rain as a combination of streak occlusion and haze-like atmospheric scattering. Similarly, Yinglong Wang et al. proposed a formulation where rain streaks and atmospheric vapors are treated as a combined transmission medium that actively absorbs and scatters light.
By modeling how vapor and streaks interact, these methods allow neural networks to simultaneously estimate the clean background, the streak geometry, and the atmospheric light, rather than treating the background as a passive layer beneath the rain.
The Role of Scene Depth
A crucial intuition behind why the rain veil behaves like fog is its relationship with scene depth. Just as in standard atmospheric haze, light traveling from more distant objects passes through a greater volume of suspended water particles, experiencing more scattering. Consequently, the veiling effect is inherently depth-dependent: foreground objects may show distinct, sharp streaks, while background objects are entirely washed out by a uniform mist.
To exploit this physical property, Xiaowei Hu et al. introduced DAF-Net, an architecture that leverages estimated scene depth to guide a neural attention mechanism. By making the network aware of depth, it can adaptively apply streak-removal filters in the foreground and dehazing operations in the background.
In the video domain, WENDING YAN et al. enforced a strict physics-based consistency between scene depth and rain transmission across consecutive frames. This depth awareness prevents the network from over-smoothing foreground textures while ensuring that the dense background veil is completely cleared.
Multi-Scale and Joint Restoration Architectures
Because heavy rain operates across multiple spatial frequencies—ranging from sharp, high-frequency foreground streaks to smooth, low-frequency background haze—network architectures must be designed to handle multi-scale features simultaneously.
For instance, the JORDER framework introduced by Wenhan Yang et al. utilizes a recurrent network architecture to jointly detect rain streak locations and estimate the accumulation layer. By iteratively refining its predictions, the network can decouple the overlapping visual artifacts.
Similarly, Huiyuan Fu et al. developed a multi-feature fusion network (MSGMFFNet) that processes pre-scaled versions of an image, using a specialized loss function to preserve edges while stripping away the misty accumulation.
Extending to Nighttime and Complex Environments
The physics of the veiling effect becomes even more complex at night or in compound weather conditions. In nighttime scenarios, the dense moisture in the air does not just reduce contrast; it actively scatters artificial light from streetlamps and headlights, creating highly non-uniform glows and flares.
To address this, Qiyuan Guan et al. introduced CST-Net, a framework that shifts processing into a learnable luminance channel to specifically model the light transport mechanisms that cause atmospheric glows.
Furthermore, as highlighted by Jie Gui et al., real-world scenarios often involve compound weather where rain, snow, and haze coexist. This necessitates sophisticated spatial-frequency modeling to separate complex light scattering from the true background, proving that robust weather removal requires networks to understand the volumetric physics of the atmosphere, not just the geometry of falling particles.
Key Foundational Work
Weighted median guided filtering method for single image rain removal (2018) — contributed 2 concept elements to this direction.
Image De-Raining Using a Conditional Generative Adversarial Network (2017) — contributed 1 concept elements to this direction.
FastDeRain: A Novel Video Rain Streak Removal Method Using Directional Gradient Priors (2018) — contributed 1 concept elements to this direction.