Adaptive Language-Aware Image Reflection Removal Network

Siyan Fang, Yuntao Wang, Jinpu Zhang, Ziwen Li, Yuehuan Wang

Introduction

Capturing clear photographs through glass surfaces often results in unwanted reflections, a phenomenon where the interior scene is superimposed over the desired exterior view. In computer vision, separating these transmission and reflection layers from a single image is a notoriously "ill-posed" problem. Recent advancements have introduced language-guided reflection removal, utilizing text descriptions to help models distinguish between layers. However, this approach faces a "hallucination" problem: the reflections themselves often confuse the captioning AI, leading to inaccurate text that can actually degrade the performance of the removal model.

To address this, we present the Adaptive Language-Aware Network (ALANet). ALANet is designed to be robust against flawed language inputs, using adaptive filtering and calibration to ensure effective reflection removal even when descriptions are incorrect or incomplete.

The ALANet Architecture

ALANet introduces three core innovations to handle the interplay between visual and textual data:

  1. Language-Aware Competition Attention Module (LCAM): This module acts as a gatekeeper. It allows language-guided attention and standard visual attention to compete. If the text description is found to be inconsistent with the visual features, the model dynamically reduces the influence of the language to prevent misguidance.
  2. Adaptive Language Calibration Module (ALCM): This optimization strategy uses visual features to "calibrate" the language features. By aligning the text representation with the actual pixel data, the model ensures a more consistent understanding of the scene.
  3. Language-Guided Spatial-Channel Cross Transformer (LSCT): This component uses semantic cues from the text to decouple information within the image's feature maps, helping the network precisely unmix transmission and reflection layers across different spatial regions.

Dataset and the "Stress Test"

Beyond standard benchmarks, this research introduces the CRLAV (Complex Reflection and Language Accuracy Variance) dataset. CRLAV contains 600 real-world image pairs and is specifically designed to test model robustness. It includes language descriptions categorized by their level of inaccuracy: Incorrect, Confused, or Incomplete. This allows for a rigorous "stress test" of how models behave when fed "bad advice" from a language model.

Performance of ALANet under various language input conditions (Incorrect, Confused, Incomplete, Accurate) compared to no language guidance, demonstrating its robustness to language variance.

Key Results and Discussion

ALANet achieves state-of-the-art performance across multiple public datasets, including a PSNR of 24.74 and SSIM of 0.868.

  • Robustness: Most notably, on the CRLAV dataset, ALANet maintained a PSNR of 19.06 even when provided with entirely incorrect language. In contrast, other language-based models saw significant performance drops when faced with inaccurate text.
  • Efficiency: The model is relatively lightweight with 8.69M parameters, making it significantly more efficient than several high-performing competitors (some exceeding 70M parameters).
  • Quantifying Hallucination: The study confirmed that reflections cause a 30-55% drop in captioning quality, proving that adaptive models like ALANet are essential for real-world applications where perfect text input cannot be guaranteed.

Conclusion and Future Work

ALANet demonstrates that language can be a powerful tool for reflection removal if handled adaptively. While the model excels at complex unmixing, extremely high-intensity reflections that saturate the sensor remain a challenge. Future research will focus on scaling the CRLAV dataset and optimizing the architecture for real-time performance on mobile devices, where reflection-heavy photography is most common.

Summary updated

Abstract

Existing image reflection removal methods struggle to handle complex reflections. Accurate language descriptions can help the model understand the image content to remove complex reflections. However, due to blurred and distorted interferences in reflected images, machine-generated language descriptions of the image content are often inaccurate, which harms the performance of language-guided reflection removal. To address this, we propose the Adaptive Language-Aware Network (ALANet) to remove reflections even with inaccurate language inputs. Specifically, ALANet integrates both filtering and optimization strategies. The filtering strategy reduces the negative effects of language while preserving its benefits, whereas the optimization strategy enhances the alignment between language and visual features. ALANet also utilizes language cues to decouple specific layer content from feature maps, improving its ability to handle complex reflections. To evaluate the model's performance under complex reflections and varying levels of language accuracy, we introduce the Complex Reflection and Language Accuracy Variance (CRLAV) dataset. Experimental results demonstrate that ALANet surpasses state-of-the-art methods for image reflection removal. The code and dataset are available at https://github.com/fashyon/ALANet.
BibTeX
@inproceedings{fang2025adaptive,
  author = {Siyan Fang and Yuntao Wang and Jinpu Zhang and Ziwen Li and Yuehuan Wang},
  title = {{Adaptive Language-Aware Image Reflection Removal Network}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/109},
}
Figure