Introduction
Capturing clear photographs through glass surfaces often results in unwanted reflections, a phenomenon where the interior scene is superimposed over the desired exterior view. In computer vision, separating these transmission and reflection layers from a single image is a notoriously "ill-posed" problem. Recent advancements have introduced language-guided reflection removal, utilizing text descriptions to help models distinguish between layers. However, this approach faces a "hallucination" problem: the reflections themselves often confuse the captioning AI, leading to inaccurate text that can actually degrade the performance of the removal model.
To address this, we present the Adaptive Language-Aware Network (ALANet). ALANet is designed to be robust against flawed language inputs, using adaptive filtering and calibration to ensure effective reflection removal even when descriptions are incorrect or incomplete.
The ALANet Architecture
ALANet introduces three core innovations to handle the interplay between visual and textual data:
- Language-Aware Competition Attention Module (LCAM): This module acts as a gatekeeper. It allows language-guided attention and standard visual attention to compete. If the text description is found to be inconsistent with the visual features, the model dynamically reduces the influence of the language to prevent misguidance.
- Adaptive Language Calibration Module (ALCM): This optimization strategy uses visual features to "calibrate" the language features. By aligning the text representation with the actual pixel data, the model ensures a more consistent understanding of the scene.
- Language-Guided Spatial-Channel Cross Transformer (LSCT): This component uses semantic cues from the text to decouple information within the image's feature maps, helping the network precisely unmix transmission and reflection layers across different spatial regions.
Dataset and the "Stress Test"
Beyond standard benchmarks, this research introduces the CRLAV (Complex Reflection and Language Accuracy Variance) dataset. CRLAV contains 600 real-world image pairs and is specifically designed to test model robustness. It includes language descriptions categorized by their level of inaccuracy: Incorrect, Confused, or Incomplete. This allows for a rigorous "stress test" of how models behave when fed "bad advice" from a language model.
Key Results and Discussion
ALANet achieves state-of-the-art performance across multiple public datasets, including a PSNR of 24.74 and SSIM of 0.868.
- Robustness: Most notably, on the CRLAV dataset, ALANet maintained a PSNR of 19.06 even when provided with entirely incorrect language. In contrast, other language-based models saw significant performance drops when faced with inaccurate text.
- Efficiency: The model is relatively lightweight with 8.69M parameters, making it significantly more efficient than several high-performing competitors (some exceeding 70M parameters).
- Quantifying Hallucination: The study confirmed that reflections cause a 30-55% drop in captioning quality, proving that adaptive models like ALANet are essential for real-world applications where perfect text input cannot be guaranteed.
Conclusion and Future Work
ALANet demonstrates that language can be a powerful tool for reflection removal if handled adaptively. While the model excels at complex unmixing, extremely high-intensity reflections that saturate the sensor remain a challenge. Future research will focus on scaling the CRLAV dataset and optimizing the architecture for real-time performance on mobile devices, where reflection-heavy photography is most common.
Summary updated