TL;DR
CLEAR is a deterministic, single-step latent framework for reflection removal that leverages pretrained diffusion priors without the need for slow, iterative sampling. Its core innovation, the Hybrid Feature Alignment Module (HFAM), solves a long-standing dilemma in the field: how to restore sharp details from the input image without accidentally reintroducing the very reflections the model just worked to remove.
What Problem Does This Paper Address?
Single Image Reflection Removal (SIRR) is a classic "ill-posed" problem in computer vision. When you take a photo through a window, the captured image is a mixture of the background scene (transmission) and the mirrored scene (reflection). Mathematically, there are infinite ways to split one image into two layers, so models must rely on strong "priors"—knowledge of what natural, clean images look like.
Recent state-of-the-art methods use pretrained generative models (like Stable Diffusion) as these priors. However, they face a catch-22. To recover fine details lost during processing, these models typically use "skip connections" to pull high-frequency information directly from the original blurry/noisy input. While this works for tasks like deblurring, in reflection removal, the input's high-frequency details include the reflection's edges. Simply copying these details back into the final image often reintroduces "ghosting" artifacts and structural inconsistencies.
Methods
The authors propose CLEAR (Color-Texture LatEnt Alignment for Single Image Reflection Removal), a two-stage pipeline that operates in the compressed latent space of a Variational Autoencoder (VAE).
- Stage 1: Latent Signal Disentanglement. CLEAR uses a dual-path U-Net architecture. One branch predicts the transmission layer, while the other predicts the reflection and a "residue" term ($\Phi$) that accounts for non-linear effects like light saturation. Unlike standard diffusion models, this is a deterministic, single-forward-pass operation—no noise schedules or random sampling required.
- Stage 2: Decoder Refinement via HFAM. This is the paper's primary contribution. Instead of raw skip connections, CLEAR uses the Hybrid Feature Alignment Module (HFAM). HFAM treats input features as "contaminated evidence." It uses a learned "color query" to look at the input skips and selectively amplify textures that match the expected color and statistics of the clean transmission layer, while attenuating features that look like reflections.
Datasets
The model was trained on a massive composite dataset of 84,425 image pairs, including synthetic data (COCO, PASCAL VOC) and real-world samples. It was evaluated against six diverse real-world benchmarks:
- Nature and Real20: Small, high-quality real-world sets.
- SIR²: A large dataset (500 images) with controlled lab settings.
- DRR-C and DRR-S: Benchmarks focused on complex and sharp reflections.
- OpenRR-1k: A recent, large-scale dataset for real-world reflection removal.
Key Insights and Ideas
- The Skip Connection Dilemma: The authors identify that standard architectural choices for image restoration (like U-Net skips) are fundamentally flawed for SIRR because the "noise" (reflection) is structured and carries its own high-frequency detail.
- Alignment over Copying: By framing detail recovery as an alignment problem rather than a copying problem, CLEAR can filter out reflection edges before they reach the final output.
- Color as a Discriminator: The latent space of VAEs encodes color information in specific channel statistics. HFAM exploits this by using a color-gated attention mechanism, effectively saying, "If this texture doesn't match the color profile of my predicted clean scene, it's probably a reflection—ignore it."
- Physics-Guided Residue: Adopting the $I = T + R + \Phi$ model allows the network to handle real-world complexities like sensor saturation more gracefully than a simple linear $I = T + R$ model.
Key Results
CLEAR achieved state-of-the-art performance across nearly all metrics. Notably, it achieved the lowest LPIPS (a metric for perceptual similarity) on every single dataset tested, indicating that its results look significantly more natural to human observers.
Quantitatively, CLEAR improved the average PSNR by 0.61 dB and reduced LPIPS by 33% compared to the strongest single-timestep baseline (DAI). It is also highly efficient, processing a $768 \times 768$ image in just 0.556 seconds on an NVIDIA A6000.
Limitations and Future Work
While CLEAR is powerful, the authors note a few limitations:
- Hallucination: Like all generative-prior models, it can "hallucinate" details in areas where the reflection is so opaque that the background is completely invisible.
- Latent Ceiling: Operating in a compressed VAE latent space imposes a hard cap on the maximum level of fine detail the model can recover.
- Future Directions: The authors suggest extending this alignment principle to multi-cue settings (like flash/no-flash photography) and exploring parameter-efficient fine-tuning (LoRA) to reduce the memory footprint during training.
Summary updated