CLEAR: Color-Texture Latent Alignment for Single Image Reflection Removal

Tu Vo, Sheir A. Zaheer, Chan Y. ParkarXiv 2026 · arXiv

TL;DR

CLEAR is a deterministic, single-step latent framework for reflection removal that leverages pretrained diffusion priors without the need for slow, iterative sampling. Its core innovation, the Hybrid Feature Alignment Module (HFAM), solves a long-standing dilemma in the field: how to restore sharp details from the input image without accidentally reintroducing the very reflections the model just worked to remove.

What Problem Does This Paper Address?

Single Image Reflection Removal (SIRR) is a classic "ill-posed" problem in computer vision. When you take a photo through a window, the captured image is a mixture of the background scene (transmission) and the mirrored scene (reflection). Mathematically, there are infinite ways to split one image into two layers, so models must rely on strong "priors"—knowledge of what natural, clean images look like.

Recent state-of-the-art methods use pretrained generative models (like Stable Diffusion) as these priors. However, they face a catch-22. To recover fine details lost during processing, these models typically use "skip connections" to pull high-frequency information directly from the original blurry/noisy input. While this works for tasks like deblurring, in reflection removal, the input's high-frequency details include the reflection's edges. Simply copying these details back into the final image often reintroduces "ghosting" artifacts and structural inconsistencies.

Methods

The authors propose CLEAR (Color-Texture LatEnt Alignment for Single Image Reflection Removal), a two-stage pipeline that operates in the compressed latent space of a Variational Autoencoder (VAE).

Figure 1. Pipeline overview. Stage 1 (latent disentanglement): the input is encoded by a frozen SDv2 VAE [30] and processed by two U-Net branches that exchange information via latent cross-attention, predicting transmission and reflection/residue latents in a single deterministic step. Stage 2 (feature alignment): the transmission latent is decoded by a cross-latent decoder augmented at every scale by the HFAM, which injects high-frequency detail and chromatic cues. During training the HFAM is supervised with ground-truth features extracted by the same frozen encoder. Snowflakes mark frozen modules; trainable modules are unmarked.
  1. Stage 1: Latent Signal Disentanglement. CLEAR uses a dual-path U-Net architecture. One branch predicts the transmission layer, while the other predicts the reflection and a "residue" term ($\Phi$) that accounts for non-linear effects like light saturation. Unlike standard diffusion models, this is a deterministic, single-forward-pass operation—no noise schedules or random sampling required.
  2. Stage 2: Decoder Refinement via HFAM. This is the paper's primary contribution. Instead of raw skip connections, CLEAR uses the Hybrid Feature Alignment Module (HFAM). HFAM treats input features as "contaminated evidence." It uses a learned "color query" to look at the input skips and selectively amplify textures that match the expected color and statistics of the clean transmission layer, while attenuating features that look like reflections.
Figure 2. Detailed view of the HFAM block, which uses color-gated attention to align input skips with clean transmission statistics.

Datasets

The model was trained on a massive composite dataset of 84,425 image pairs, including synthetic data (COCO, PASCAL VOC) and real-world samples. It was evaluated against six diverse real-world benchmarks:

  • Nature and Real20: Small, high-quality real-world sets.
  • SIR²: A large dataset (500 images) with controlled lab settings.
  • DRR-C and DRR-S: Benchmarks focused on complex and sharp reflections.
  • OpenRR-1k: A recent, large-scale dataset for real-world reflection removal.

Key Insights and Ideas

  • The Skip Connection Dilemma: The authors identify that standard architectural choices for image restoration (like U-Net skips) are fundamentally flawed for SIRR because the "noise" (reflection) is structured and carries its own high-frequency detail.
  • Alignment over Copying: By framing detail recovery as an alignment problem rather than a copying problem, CLEAR can filter out reflection edges before they reach the final output.
  • Color as a Discriminator: The latent space of VAEs encodes color information in specific channel statistics. HFAM exploits this by using a color-gated attention mechanism, effectively saying, "If this texture doesn't match the color profile of my predicted clean scene, it's probably a reflection—ignore it."
  • Physics-Guided Residue: Adopting the $I = T + R + \Phi$ model allows the network to handle real-world complexities like sensor saturation more gracefully than a simple linear $I = T + R$ model.

Key Results

CLEAR achieved state-of-the-art performance across nearly all metrics. Notably, it achieved the lowest LPIPS (a metric for perceptual similarity) on every single dataset tested, indicating that its results look significantly more natural to human observers.

Figure 3. Visual comparison against all methods on SIR2 [39] (top two rows) and Nature [21] (bottom two rows). Our method yields cleaner separation and recovers structural detail and texture more faithfully. Zoom in for detail.

Quantitatively, CLEAR improved the average PSNR by 0.61 dB and reduced LPIPS by 33% compared to the strongest single-timestep baseline (DAI). It is also highly efficient, processing a $768 \times 768$ image in just 0.556 seconds on an NVIDIA A6000.

Limitations and Future Work

While CLEAR is powerful, the authors note a few limitations:

  • Hallucination: Like all generative-prior models, it can "hallucinate" details in areas where the reflection is so opaque that the background is completely invisible.
  • Latent Ceiling: Operating in a compressed VAE latent space imposes a hard cap on the maximum level of fine detail the model can recover.
  • Future Directions: The authors suggest extending this alignment principle to multi-cue settings (like flash/no-flash photography) and exploring parameter-efficient fine-tuning (LoRA) to reduce the memory footprint during training.

Summary updated

Abstract

Single Image Reflection Removal (SIRR) seeks to recover the transmitted scene from a photograph captured through glass. Recent latent restoration methods use pretrained generative priors, but many recover detail by feeding high-frequency features from the degraded input back into the decoder. This works for degradations such as blur, noise, or haze, where input detail mostly belongs to the scene. It is less suitable for SIRR, where reflections add a second structured layer: directly reusing the input's skip connections can sharpen the image while also bringing reflection edges back. Latent compression also introduces color drift and fine-detail loss. We propose CLEAR (Color-Texture LatEnt Alignment for Single Image Reflection Removal), a deterministic, single-timestep, non-iterative latent framework that uses a pretrained latent diffusion backbone as a natural-image prior. A physics-guided separator organizes the prediction around the transmission, reflection, and residue formation model and outputs coarse layer latents in one forward pass. The Hybrid Feature Alignment Module (HFAM) then treats input-derived skips as contaminated layer-mixture features. It rectifies their high-frequency content with a clean-transmission-aligned color query, learned from the input features themselves under ground-truth feature supervision, keeping transmission-consistent texture while attenuating reflection-colored detail. CLEAR achieves the best average across fidelity and perceptual metrics on six real-world benchmarks and the lowest LPIPS on every dataset, with the largest gains on the hardest benchmarks.
BibTeX
@misc{vo2026clear,
  author = {Tu Vo and Sheir A. Zaheer and Chan Y. Park},
  title = {{CLEAR: Color-Texture Latent Alignment for Single Image Reflection Removal}},
  year = {2026},
  eprint = {2611.05969},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.05969},
}
Figure