Abstract
Single image deraining often struggles with the domain gap between synthetic training data and complex real-world environments. We present a high-order collaborative network architecture that leverages multi-scale constraints and bidirectional similarity mining to extract robust features. To bridge the simulation-to-reality gap, we introduce a self-supervised "online-update" mechanism. Our approach achieves state-of-the-art performance on synthetic benchmarks while demonstrating superior generalization to real-world rainy images with significantly fewer parameters than competing models.
Introduction
Single image deraining is a fundamental inverse problem in computer vision, aiming to recover a clean background $B$ from a rainy observation $O = B + R$. Despite the success of deep learning, existing methods often suffer from feature underutilization and a significant domain gap when transitioning from synthetic datasets to real-world scenarios. Many models treat the deraining process as a black box, ignoring the structural information available at intermediate layers and across different image scales. Furthermore, models trained on synthetic rain often fail to handle the diverse densities and directions of natural rain.
Methodology
Our proposed framework addresses these challenges through three key innovations:
High-Order Collaborative Networks
Instead of a standard cascaded architecture, we employ three sub-networks (Bottom, Middle, and Up) that operate collaboratively. This design facilitates multi-stage self-supervision and bidirectional feature flow, ensuring that shallow and deep features are effectively integrated. The authors found that this collaborative flow is more effective than simply stacking layers.
Multi-Scale Compact Constraints (MSCC)
To regularize the network's feature space, we implement constraints based on a Laplacian pyramid. By requiring intermediate layers to reconstruct multi-scale versions of the clean image, the network is forced to learn structured, edge-aware features that are crucial for distinguishing rain streaks from background textures.
Bidirectional Scale-Content Similarity Mining (BiSCSM)
Recognizing the repetitive nature of rain streaks across scales, the BiSCSM module performs patch-based similarity searches in both "Down-to-Up" and "Up-to-Down" directions. This allows the model to capture long-range dependencies and exploit cross-scale redundancies more effectively than unidirectional methods.
Online-Update Learning
To handle real-world data without ground truth, we introduce an online-update strategy. Given a real rainy image, the model generates an initial prediction as a "pseudo-label" and iteratively fine-tunes itself using a KL-Divergence loss. This ensures background consistency while progressively refining the rain removal process for the specific characteristics of the target image.
Experimental Results
The model was evaluated on five synthetic benchmarks (Rain200H/L, Rain1200, Rain1400, and Rain12) and various real-world datasets.
Performance and Efficiency
Our method consistently outperformed 11 state-of-the-art (SOTA) methods. On the challenging Rain200H dataset, it achieved a PSNR of 29.985 dB. Notably, our model maintains high performance with a compact footprint of only 2.04M parameters, making it significantly more efficient than models like MSPFN (21M) or DCSFN (6.45M).
Qualitative Analysis
Visual comparisons demonstrate that our approach excels at preserving fine details and textures that are often blurred by other methods. The collaborative architecture and multi-scale constraints allow for precise rain removal without sacrificing image sharpness, as seen in the comparison between our results and other SOTA methods like MPRNet.
Conclusion
This paper introduces a collaborative, multi-scale approach to single image deraining that effectively bridges the gap between synthetic training and real-world application. By combining high-order feature flow with an online-update mechanism, we achieve a superior balance of performance and efficiency. Future work will focus on reducing the computational overhead of the online-update process to enable real-time deployment.
Summary updated