Abstract
Removing raindrops from images taken through glass is a challenging yet critical task for outdoor vision systems. This paper presents a novel Convolutional Neural Network (CNN) that leverages a Joint Physical Shape + Channel Attention (JPCA) module to restore image clarity. By incorporating geometric priors—specifically the roundness and closedness of raindrops—the model effectively identifies and removes distortions while preserving background details. Experimental results on benchmark datasets demonstrate that this approach outperforms existing state-of-the-art methods, including GAN-based architectures, in both quantitative metrics and visual quality.
Introduction
Adherent raindrops on windows or camera lenses pose a significant challenge for computer vision applications, particularly in autonomous driving and surveillance. These droplets act as complex lenses, refracting and reflecting light, which leads to severe visual distortions or complete occlusion of the background scene.
Restoring a clean image from a single corrupted frame is a "highly ill-posed" problem. Traditional methods often fail to generalize across varying raindrop sizes and shapes, frequently leaving behind artifacts or failing to detect transparent droplets. This paper addresses these issues by introducing a shape-driven attention mechanism that utilizes the physical properties of raindrops to guide the restoration process.
Methodology: Joint Physical Shape + Channel Attention
The proposed architecture is an encoder-decoder CNN enhanced by the Joint Physical Shape + Channel Attention (JPCA) module. This module integrates two distinct types of attention to improve feature extraction and image reconstruction.
1. Shape-Driven (Physical) Attention
Unlike standard attention mechanisms that learn weights purely from data, the shape-driven module incorporates a "torque operator" to search for geometric features characteristic of raindrops: convexness and contour closedness. Since raindrops are typically elliptical or round, these physical priors allow the network to accurately locate and mask raindrop regions. This is achieved through specialized convolutional kernels designed to learn elliptical parameters.
2. Channel Re-calibration (Channel Attention)
To complement spatial attention, the model employs a channel re-calibration module (similar to Squeeze-and-Excitation blocks). This component assigns weights to different feature channels, enabling the network to prioritize background textures over raindrop-related artifacts.
3. Robust Edge Detection
To support the shape-driven module, the authors implemented a custom edge detector. This detector is more resilient to environmental noise than standard gradient-based methods, ensuring reliable droplet boundary detection even in low-contrast or poorly lit scenes.
Experimental Setup and Datasets
The model was rigorously tested using the following:
- Qian et al. Dataset: A benchmark consisting of 1,110 image pairs (raindrop-corrupted vs. clean ground truth), created by photographing scenes through glass before and after applying water spray.
- Real-World Testing: The model was also evaluated on "wild" images captured via mobile phones in actual rainy conditions to verify its generalization capabilities beyond controlled environments.
Results and Discussion
The proposed JPCA-based model was compared against several leading methods, including Eigen et al.’s CNN and the GAN-based AttentGAN.
Quantitative Performance
On the Qian et al. benchmark, the proposed model achieved a Peak Signal-to-Noise Ratio (PSNR) of 30.86 dB and a Structural Similarity Index (SSIM) of 0.9263. These results surpass AttentGAN, which recorded 30.55 dB and 0.9023, respectively. Ablation studies confirmed the importance of the JPCA module; its removal led to a nearly 1 dB drop in PSNR.
Qualitative Comparison
Visual analysis shows that the proposed method produces significantly cleaner images with fewer "ghosting" artifacts or blurry patches than previous state-of-the-art techniques.
Interestingly, the authors found that their targeted CNN approach outperformed Generative Adversarial Networks (GANs). While GANs are known for producing visually "realistic" textures, they often introduce hallucinated artifacts in the context of raindrop removal, whereas the proposed CNN maintains higher fidelity to the actual background.
Conclusion and Future Work
This research demonstrates that integrating physical priors into deep learning architectures can significantly enhance performance for specific image restoration tasks. By combining shape-driven and channel attention, the model effectively "sees through" raindrops.
Future work will focus on:
- Handling Opaque Occlusions: Improving the reconstruction of areas completely blocked by opaque reflections.
- Computational Efficiency: Optimizing multi-scale kernels for real-time deployment on automotive hardware.
- Temporal Consistency: Extending the framework to video sequences to ensure stable, flicker-free restoration in dynamic environments.
Summary updated