Sequential Layer Decomposition for Image Restoration

18 concepts12 papers1 proposal

Image restoration tasks—such as removing glass reflections, clearing nighttime haze, or brightening low-light photos—are fundamentally ill-posed inverse problems. A single degraded pixel is the composite of multiple overlapping physical phenomena: background light, foreground reflection, atmospheric scattering, and sensor noise.

Traditional deep learning approaches often attempt to map this messy input directly to a clean output in a single step. However, this unified approach suffers from a compounding error trap. For instance, when a network tries to brighten a low-light image, it inadvertently amplifies the sensor noise hidden in the shadows. When it tries to remove haze, it struggles with concurrent artificial color casts.

To break this deadlock, a powerful paradigm has emerged: sequential layer decomposition. Instead of solving all degradation factors simultaneously, these methods decouple the image into its constituent physical layers (such as illumination and reflectance, or reflection and transmission) and estimate them in a deliberate, multi-stage sequence. By fixing or isolating one component first, the network prevents mutual interference, simplifies the optimization landscape, and traps unwanted artifacts or noise in a specific layer where they can be safely neutralized.


Trapping Noise in Low-Light Enhancement

Low-light image enhancement is a classic double-edged sword. According to Retinex theory, an image $I$ can be decomposed into a high-frequency reflectance map $R$ (the intrinsic details of the scene) and a smooth illumination map $L$ (the light falling upon it), such that $I = R \times L$. When an algorithm brightens a dark photo, it essentially scales up the illumination component. However, doing so directly amplifies the high-frequency sensor noise hidden in the dark pixels.

[Degraded Low-Light Image]
          │
          ▼
┌──────────────────────────────────┐
│   Stage 1: Illumination Map (L)  │ ──► Smooth, noise-free lighting
└──────────────────────────────────┘
          │
          ▼
┌──────────────────────────────────┐
│    Stage 2: Reflectance Map (R)  │ ──► Traps and isolates sensor noise
└──────────────────────────────────┘
          │
          ▼
┌──────────────────────────────────┐
│    Targeted Denoising on (R)     │ ──► Preserves fine structural textures
└──────────────────────────────────┘

Sequential decomposition solves this by changing the order of operations. By estimating the smooth illumination map first, the optimization process isolates the sensor noise entirely within the reflectance layer. As pioneered by Chen Wei et al. in Deep Retinex Decomposition for Low-Light Enhancement, separating these components allows networks to perform targeted denoising operations strictly on the reflectance map, preventing the noise amplification that plagues unified frameworks.

Why must this happen sequentially rather than simultaneously? As explained by Tingting Wu et al. in Retinex Image Enhancement Based on Sequential Decomposition With a Plug-and-Play Framework and Xutong Ren et al. in Joint Enhancement and Denoising Method via Sequential Decomposition, traditional alternating optimization methods suffer from mutual interference—the network oscillates between updating illumination and reflectance, causing noise to pollute the illumination boundaries. Estimating the illumination first fixes the structural boundaries, ensuring that noise is cleanly trapped in the reflectance layer.

Once the noise is isolated, downstream restoration becomes significantly more effective. Chenglin Wen et al. in A depth iterative illumination estimation network for low-light image enhancement based on retinex theory leverage this by applying residual learning specifically to the reflectance component, isolating structural textures from sensor noise. Similarly, Aline Roumy et al. in Illumination colorée et information commune pour le modèle Horn-Retinex en restauration d'images à faible luminosité demonstrate that this separation preserves fine details and textures that would otherwise be blurred by a global denoiser.

This strategy is not limited to static images or convolutional networks; Jiafei Wu et al. in Seeing Dynamic Scene in the Dark: A High-Quality Video Dataset with Mechatronic Alignment show that separating noise estimation from illumination mapping prevents noise amplification in dynamic video enhancement, while Chunming He et al. in Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Model extend this concept to generative latent diffusion spaces to achieve robust color correction without losing thin structures.


Reflection-First Layer Separation

A similar physical coupling occurs when photographing through glass. The captured image $M$ is a linear combination of the background transmission $T$ and the foreground reflection $R$ ($M = R + T$). Intuitively, one might design a network to predict the background $T$ directly. However, reflections often contain high-frequency details and strong illumination that mimic actual background objects, confusing the network.

[Mixed Image (M = R + T)] ──► [Stage 1: Estimate Reflection (R)] ──► [Stage 2: Reconstruct Transmission (T)]

To solve this, researchers flipped the problem on its head: estimate the reflection layer first. In Polarized Reflection Removal With Perfect Alignment in the Wild, Chenyang Lei et al. introduced a two-stage pipeline where a dedicated module first isolates the reflection layer. This estimated reflection is then used as a conditional input for a second module to reconstruct the background transmission. Explicitly modeling the interference of the reflection first allows the model to subtract its influence before recovering the underlying scene.

This "reflection-first" sequential approach becomes crucial when using auxiliary hardware cues like flash photography. While a flash can help identify reflections, it introduces harsh shadows, illuminated dust, and color distortions. As demonstrated by the authors in Robust Reflection Removal With Reflection-Free Flash-Only Cues and later in Robust Reflection Removal with Flash-only Cues in the Wild, predicting the reflection component as an intermediate step prevents these flash-related artifacts from leaking into the final background image. The model uses the flash cue to isolate the unwanted reflection layer, filtering out lighting defects so that the final transmission output retains the natural ambient lighting of the original scene.


Disentangling Environmental Degradation

The power of sequential decomposition extends to scattering media, such as haze or underwater environments, where degradation is caused by a mixture of spatial attenuation and environmental lighting.

In nighttime dehazing, for example, networks must contend with both uniform atmospheric haze and highly non-uniform artificial color casts from streetlights. In Nighttime Dehazing with a Synthetic Benchmark, Jing Zhang et al. demonstrate that addressing these two phenomena simultaneously leads to severe visual artifacts. By structuring the restoration as a sequential pipeline that corrects the color cast before attempting haze removal, they prevent the errors of one process from compounding the other in complex urban lighting conditions.

[Nighttime Hazy Image] ──► [Stage 1: Color Cast Correction] ──► [Stage 2: Haze Removal] ──► [Clean Scene]

Similarly, zero-shot restoration frameworks leverage this structural separation to operate without paired training data. In Zero-Shot Single Image Restoration Through Controlled Perturbation of Koschmieder's Model, Aupendu Kar et al. employ two distinct sub-networks to independently estimate the pixel-wise transmission map and the global atmospheric light. This explicit architectural decoupling allows the model to separate the spatial attenuation of the scene from the uniform environmental illumination, preventing the network from confusing a localized loss of contrast with a global lighting change.


Why the Sequential Strategy Works

The consistent success of sequential layer decomposition across low-light enhancement, reflection removal, and dehazing boils down to two core mathematical and architectural intuitions:

  1. Simplifying the Optimization Landscape: Jointly estimating multiple coupled variables (like reflectance and illumination, or transmission and reflection) creates a highly non-convex optimization space with severe local minima. Conflicting gradients—such as trying to boost contrast while suppressing noise—cause networks to output blurred or artifact-laden results. Sequential decomposition breaks a complex joint inverse problem into a feed-forward chain of simpler, well-posed sub-problems.
  2. Asymmetric Error Isolation: In these physical models, one layer is naturally smoother or more constrained than the other (e.g., illumination maps are spatially smooth, reflections are often isolated or predictable via polarization/flash cues). By forcing the network to estimate the more constrained layer first, the remaining unconstrained degradation (like sensor noise or flash artifacts) is mathematically forced into the secondary layer. This allows targeted, domain-specific restoration networks to clean up the artifacts without destroying the structural integrity of the primary scene.

Key Foundational Work

Joint Enhancement and Denoising Method via Sequential Decomposition (2018) — contributed 3 concept elements to this direction.
Nighttime Dehazing with a Synthetic Benchmark (2020) — contributed 2 concept elements to this direction.
Deep Retinex Decomposition for Low-Light Enhancement (2018) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_sequential_reflection_transmission_estim_de290bf3,
  title = {sequential reflection-transmission estimation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/sequential-reflection-transmission-estimation/rdr_de290bf3f416462d80714ec4333c1d7b},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_sequential_reflection_transmission_estim_de290bf3_v1,
  title = {sequential reflection-transmission estimation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/sequential-reflection-transmission-estimation/rdr_de290bf3f416462d80714ec4333c1d7b/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake