Abstract
In the field of multimedia, single image deraining is a basic pre-processing work, which can greatly improve the visual effect of subsequent high-level tasks in rainy conditions. In this paper, we propose an effective algorithm, called JDNet, to solve the single image deraining problem and conduct the segmentation and detection task for applications. Specifically, considering the important information on multi-scale features, we propose a Scale-Aggregation module to learn the features with different scales. Simultaneously, Self-Attention module is introduced to match or outperform their convolutional counterparts, which allows the feature aggregation to adapt to each channel. Furthermore, to improve the basic convolutional feature transformation process of Convolutional Neural Networks (CNNs), Self-Calibrated convolution is applied to build long-range spatial and inter-channel dependencies around each spatial location that explicitly expand fields-of-view of each convolutional layer through internal communications and hence enriches the output features. By designing the Scale-Aggregation and Self-Attention modules with Self-Calibrated convolution skillfully, the proposed model has better deraining results both on real-world and synthetic datasets. Extensive experiments are conducted to demonstrate the superiority of our method compared with state-of-the-art methods.
1. Introduction
Single-image deraining is a classic ill-posed problem in computer vision: reconstructing a clean background from an image obscured by stochastic rain streaks. While deep learning has advanced this field, standard Convolutional Neural Networks (CNNs) often suffer from stationary weights that ignore image content, sensitivity to varying rain scales, and limited receptive fields that struggle with long-range context.
To address these issues, we introduce JDNet (Joint Deraining Network), a novel architecture that combines content-adaptive attention with multi-scale feature aggregation. By focusing on selective filtering and internal communication between layers, JDNet effectively removes complex rain patterns while preserving the structural integrity of the underlying scene.
2. Methodology
The core of JDNet is a "joint unit" repeated through dense connections, incorporating three primary innovations:
2.1 Pairwise Self-Attention (PSA)
Unlike standard convolutions with fixed kernels, the PSA module calculates weights based on the relationship between a pixel and its $7 \times 7$ neighborhood. This allows the network to adapt its filtering strategy dynamically based on whether it is processing a rain streak or a fine background texture, ensuring content adaptability.
2.2 Scale-Aggregation (SA)
Rain streaks vary significantly in size and density. The SA module utilizes a pyramid-like structure to downsample features to multiple scales, process them through Residual Blocks to learn scale-specific patterns, and then upsample and concatenate them. This ensures the model can handle everything from fine mist to large, distinct streaks.
2.3 Self-Calibrated (SC) Convolution
SC convolutions split feature channels and process a subset at a lower resolution to expand the receptive field. This "calibrated" information guides the feature transformation at the original resolution, enabling the model to build long-range spatial dependencies and enrich output features through internal communication without a substantial increase in parameters.
3. Experimental Results
JDNet was evaluated against state-of-the-art methods on synthetic benchmarks (Rain100H, Rain100L, Rain12) and real-world datasets.
3.1 Quantitative Performance
On the challenging Rain100H dataset, JDNet achieved a PSNR of 30.02 dB and an SSIM of 0.92, significantly outperforming the previous leading method, PreNet (27.89 dB / 0.89). Ablation studies confirmed that each component—PSA, SA, and SC—is essential to this performance gain.
3.2 Qualitative Analysis and Downstream Tasks
Visual comparisons demonstrate that JDNet produces cleaner backgrounds with fewer artifacts than competing models, preserving sharp edges where others over-smooth. Furthermore, the paper validates the utility of JDNet as a pre-processing step; using JDNet for deraining significantly improved the accuracy of semantic segmentation (via PSPNet) and object detection in rainy conditions.
4. Conclusion
JDNet provides a robust solution for single-image deraining by integrating content-adaptive self-attention, multi-scale aggregation, and self-calibrated convolutions. The model's ability to preserve fine textures while removing heavy rain streaks makes it highly effective for both human visualization and automated downstream vision tasks. Future work will focus on optimizing computational complexity for mobile deployment and addressing extreme weather combinations like rain and heavy fog.
Summary updated