Rain Streak Removal via Dual Graph Convolutional Network

Xueyang Fu, Qi Qi, Zheng-Jun Zha, Yurui Zhu, Xinghao Ding

Introduction

Single-image de-raining is a challenging "ill-posed" problem where an algorithm must distinguish between actual scene details and overlapping rain streaks. Most modern solutions rely on Deep Convolutional Neural Networks (CNNs). However, standard CNNs are inherently local operators, processing small neighborhoods of pixels at a time.

Rain streaks are often spatially long and complex, requiring a global perspective to identify patterns across the entire image. While "non-local" modules can capture these long-range dependencies, they are typically memory-intensive and slow. This paper introduces a Dual Graph Convolutional Network (GCN) designed to capture global context—both spatially and across feature channels—without the massive computational overhead of traditional non-local methods.

Methodology

The proposed architecture utilizes a symmetrically skip-connected structure containing a "Basic Unit" repeated 11 times. This unit is designed to synergize local and global information through three primary components:

  1. Spatial GCN (sGCN): This module treats pixels as nodes in a graph, allowing every pixel to communicate with every other pixel regardless of distance. By employing a matrix re-ordering trick based on the associative rule, the authors reduce computational complexity from $O((HW)^2)$ to $O(HW)$, making it significantly faster than standard non-local modules.
  2. Dilated Convolutional Module (DCM): To handle rain streaks of varying sizes, the DCM uses parallel paths of dilated convolutions. This enables the network to capture local features across multiple receptive fields (ranging from 3x3 to 13x13) simultaneously.
  3. Channel GCN (cGCN): While the spatial GCN focuses on positional relationships, the channel GCN models interdependencies between different feature channels. This helps the network suppress rain-related features while preserving essential scene details.
The proposed Dual GCN architecture and its internal Basic Unit, which integrates spatial GCN, dilated convolutions, and channel GCN modules.

Experimental Evaluation

The model was evaluated on several widely recognized benchmarks, including synthetic datasets (Rain12, Rain200L/H, DDN-Data, and DID-Data) and the large-scale real-world SPA-Data set, which contains approximately 640,000 image pairs. The network was trained end-to-end using a Mean Absolute Error (MAE) loss function to balance rain removal with detail preservation.

Key Results and Insights

Quantitative and Qualitative Performance

The Dual GCN model consistently outperformed 14 baseline methods (including NLEDN, RESCAN, and PReNet) across all benchmark datasets. On the challenging Rain200H dataset, it achieved a PSNR of 31.15 dB. Visually, the model excels at preserving fine textures—such as fabric weaves or distant architectural edges—that competing models often blur during the de-raining process.

Visual comparison of the proposed method against state-of-the-art de-raining models across multiple datasets, demonstrating superior texture preservation.

Efficiency and Orthogonal Modeling

A key insight of this work is the use of orthogonal global modeling. By addressing spatial and channel relations separately but within the same framework, the network gains a richer representation of the image. Furthermore, the authors demonstrate that GCNs, when implemented efficiently, provide a superior alternative to self-attention-style modules for non-local operations in computer vision.

Conclusion

The Dual GCN framework provides a robust and efficient solution for single-image de-raining by effectively combining local multi-scale convolutions with global graph reasoning. While currently focused on static images, the authors suggest that this framework could be extended to video de-raining or adapted for other low-level vision tasks like denoising and super-resolution. Future work may also address extreme weather conditions, such as rain accumulation and mist, which require additional dehazing components.

Summary updated

Abstract

Deep convolutional neural networks (CNNs) have become dominant in the single image de-raining area. However, most deep CNNs-based de-raining methods are designed by stacking vanilla convolutional layers, which can only be used to model local relations. Therefore, long-range contextual information is rarely considered for this specific task. To address the above problem, we propose a simple yet effective dual graph convolutional network (GCN) for single image rain removal. Specifically, we design two graphs to perform global relational modeling and reasoning. The first GCN is used to explore global spatial relations among pixels in feature maps, while the second GCN models the global relations across the channels. Compared to standard convolutional operations, the proposed two graphs enable the network to extract representations from new dimensions. To achieve the image rain removal, we further embed these two graphs and multi-scale dilated convolution into a symmetrically skip-connected network architecture. Therefore, our dual graph convolutional network is able to well handle complex and spatially long rain streaks by exploring multiple representations, e.g., multi-scale local feature, global spatial coherence and cross-channel correlation. Meanwhile, our model is easy to implement, end-to-end trainable and computationally efficient. Extensive experiments on synthetic and real data demonstrate that our method achieves significant improvements over the recent state-of-the-art methods.
BibTeX
@inproceedings{fu2021rain,
  author = {Xueyang Fu and Qi Qi and Zheng-Jun Zha and Yurui Zhu and Xinghao Ding},
  title = {{Rain Streak Removal via Dual Graph Convolutional Network}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i2.16224},
}
Figure