Online-Updated High-Order Collaborative Networks for Single Image Deraining

Cong Wang, Jinshan Pan, Xiao-Ming Wu

Abstract

Single image deraining often struggles with the domain gap between synthetic training data and complex real-world environments. We present a high-order collaborative network architecture that leverages multi-scale constraints and bidirectional similarity mining to extract robust features. To bridge the simulation-to-reality gap, we introduce a self-supervised "online-update" mechanism. Our approach achieves state-of-the-art performance on synthetic benchmarks while demonstrating superior generalization to real-world rainy images with significantly fewer parameters than competing models.

Introduction

Single image deraining is a fundamental inverse problem in computer vision, aiming to recover a clean background $B$ from a rainy observation $O = B + R$. Despite the success of deep learning, existing methods often suffer from feature underutilization and a significant domain gap when transitioning from synthetic datasets to real-world scenarios. Many models treat the deraining process as a black box, ignoring the structural information available at intermediate layers and across different image scales. Furthermore, models trained on synthetic rain often fail to handle the diverse densities and directions of natural rain.

Methodology

Our proposed framework addresses these challenges through three key innovations:

High-Order Collaborative Networks

Instead of a standard cascaded architecture, we employ three sub-networks (Bottom, Middle, and Up) that operate collaboratively. This design facilitates multi-stage self-supervision and bidirectional feature flow, ensuring that shallow and deep features are effectively integrated. The authors found that this collaborative flow is more effective than simply stacking layers.

Multi-Scale Compact Constraints (MSCC)

To regularize the network's feature space, we implement constraints based on a Laplacian pyramid. By requiring intermediate layers to reconstruct multi-scale versions of the clean image, the network is forced to learn structured, edge-aware features that are crucial for distinguishing rain streaks from background textures.

Bidirectional Scale-Content Similarity Mining (BiSCSM)

Recognizing the repetitive nature of rain streaks across scales, the BiSCSM module performs patch-based similarity searches in both "Down-to-Up" and "Up-to-Down" directions. This allows the model to capture long-range dependencies and exploit cross-scale redundancies more effectively than unidirectional methods.

Online-Update Learning

To handle real-world data without ground truth, we introduce an online-update strategy. Given a real rainy image, the model generates an initial prediction as a "pseudo-label" and iteratively fine-tunes itself using a KL-Divergence loss. This ensures background consistency while progressively refining the rain removal process for the specific characteristics of the target image.

Experimental Results

The model was evaluated on five synthetic benchmarks (Rain200H/L, Rain1200, Rain1400, and Rain12) and various real-world datasets.

Performance and Efficiency

Our method consistently outperformed 11 state-of-the-art (SOTA) methods. On the challenging Rain200H dataset, it achieved a PSNR of 29.985 dB. Notably, our model maintains high performance with a compact footprint of only 2.04M parameters, making it significantly more efficient than models like MSPFN (21M) or DCSFN (6.45M).

Figure 1: Comparison of SSIM vs. Model Parameters on the Rain200H dataset. Our method achieves superior SSIM with significantly fewer parameters.

Qualitative Analysis

Visual comparisons demonstrate that our approach excels at preserving fine details and textures that are often blurred by other methods. The collaborative architecture and multi-scale constraints allow for precise rain removal without sacrificing image sharpness, as seen in the comparison between our results and other SOTA methods like MPRNet.

Figure 5: Visual result of our proposed method on a rainy scene, demonstrating effective rain removal and detail preservation.

Conclusion

This paper introduces a collaborative, multi-scale approach to single image deraining that effectively bridges the gap between synthetic training and real-world application. By combining high-order feature flow with an online-update mechanism, we achieve a superior balance of performance and efficiency. Future work will focus on reducing the computational overhead of the online-update process to enable real-time deployment.

Summary updated

Abstract

Single image deraining is an important and challenging task for some downstream artificial intelligence applications such as video surveillance and self-driving systems. Most of the existing deep-learning-based methods constrain the network to generate derained images but few of them explore features from intermediate layers, different levels, and different modules which are beneficial for rain streaks removal. In this paper, we propose a high-order collaborative network with multi-scale compact constraints and a bidirectional scale-content similarity mining module to exploit features from deep networks externally and internally for rain streaks removal. Externally, we design a deraining framework with three sub-networks trained in a collaborative manner, where the bottom network transmits intermediate features to the middle network which also receives shallower rainy features from the top network and sends back features to the bottom network. Internally, we enforce multi-scale compact constraints on the intermediate layers of deep networks to learn useful features via a Laplacian pyramid. Further, we develop a bidirectional scale-content similarity mining module to explore features at different scales in a down-to-up and up-to-down manner. To improve the model performance on real-world images, we propose an online-update learning approach, which uses real-world rainy images to fine-tune the network and update the deraining results in a self-supervised manner. Extensive experiments demonstrate that our proposed method performs favorably against eleven state-of-the-art methods on five public synthetic datasets and one real-world dataset. The source code will be available at \url{https://supercong94.wixsite.com/supercong94}.
BibTeX
@inproceedings{wang2022online,
  author = {Cong Wang and Jinshan Pan and Xiao-Ming Wu},
  title = {{Online-Updated High-Order Collaborative Networks for Single Image Deraining}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i2.20140},
}
Figure