When you take a photo through a window, the camera captures two distinct worlds: the scene behind the glass (the transmission) and the scene reflected on its surface. To a computer, these are simply summed together into a single pixel value. This creates a classic "ill-posed" problem where one observation must be decomposed into two unknown components ($I = T + R$). Because an infinite number of combinations of $T$ and $R$ can produce the same $I$, the challenge is not just mathematical, but physical.
The Ambiguity of Superposition
The fundamental difficulty in Single Image Reflection Removal (SIRR) is that the transmission and reflection layers often share the same visual characteristics. As noted by Han Li and Changshui Zhang in Efficient Superimposition Recovering Algorithm, this is a blind source separation problem. Without a way to distinguish which gradients belong to which layer, models often "hallucinate" or fail to remove bright, sharp reflections that look like real objects.
Early benchmarks like $SIR^2$, developed by Renjie Wan and Boxin Shi in Benchmarking Single-Image Reflection Removal Algorithms, highlighted that while algorithms might work on simple synthetic data, real-world physics—like double-paned glass or varying focal planes—introduces complexities that simple additive models cannot capture.
Breaking Symmetry with Physical Cues
To solve an ill-posed problem, researchers introduce "priors" or additional data channels that break the mathematical symmetry between the two layers.
- Polarization: Light reflected off a non-metallic surface like glass becomes polarized. Chenyang Lei and Qifeng Chen demonstrated in Polarized Reflection Removal With Perfect Alignment in the Wild that by capturing images through different polarization angles, the reflection's intensity changes while the transmission remains relatively stable. This provides the necessary "physical anchor" to unmix the layers.
- Flash Cues: A flash primarily illuminates the scene behind the glass, while the reflected environment (often far away) remains dark. Mingyang Xie leveraged this in Flash-Splat, using 3D Gaussian Splatting to synthesize flash/no-flash pairs that reveal the background's true colors.
- Parallax and Depth: If the camera moves slightly, the reflection and transmission shift at different rates due to their different depths. Simon Niklaus and Xuaner Cecilia Zhang used dual-view stereo pairs in Learned Dual-View Reflection Removal to exploit this motion parallax, effectively "seeing around" the reflection.
Generative Priors and Latent Rectification
When only a single RGB image is available, researchers turn to generative models to provide a "sense" of what a clean image should look like.
Recent work like PromptRR by Tao Wang uses Diffusion Models to generate frequency-based "visual prompts" that guide a Transformer in separating the layers. Similarly, Mingjia Li and Qiming Hu introduced GenSIRR in Rectifying Latent Space for Generative Single-Image Reflection Removal, which performs the separation in the latent space of a diffusion model. This works because the model's internal representation of "natural images" acts as a powerful filter, identifying components that don't belong to a coherent scene.
Beyond Simple Addition
The traditional model $I = T + R$ is often too simplistic. Real reflections involve non-linearities, residues, and spectral shifts. Qiang Wen addressed this in Single Image Reflection Removal Beyond Linearity by creating SynNet, which generates more realistic synthetic training data.
Furthermore, Qiming Hu and Xiaojie Guo proposed DSRNet in Single Image Reflection Separation via Component Synergy, which adds a learnable "residue" term to the equation. This allows the network to account for complex optical effects that a simple sum cannot describe, such as the slight blurring or tinting caused by the glass itself.
Semantic and Contextual Guidance
The most recent frontier involves using high-level context to identify reflections. Yuchen Hong explored this in Panoramic Image Reflection Removal, where a 360° view provides a direct look at the environment being reflected. Even without a panorama, Siyan Fang showed in Adaptive Language-Aware Image Reflection Removal Network that language descriptions (e.g., "remove the reflection of the trees") can help the model focus its attention on specific interfering structures, turning a blind mathematical problem into a guided semantic task.