Towards Universal Rainy Image Restoration: Benchmark and Baseline

Hujie YanAAAI 2025 · DOI

Introduction

In the field of computer vision, image deraining has traditionally been treated as a set of specialized sub-tasks. Models are often optimized for specific degradations, such as thin rain streaks, large raindrops on a lens, or the "veiling effect" of rain accumulation. However, real-world scenarios—particularly in autonomous driving—frequently involve a complex mixture of these patterns. Existing methods often struggle with "distribution shifts" when encountering rain types outside their training data. To address this, researchers have introduced the task of Universal Rainy Image Restoration (URIR), aiming for a single model capable of handling diverse rain conditions simultaneously.

The URIR-8K Benchmark

A significant hurdle in universal restoration is the lack of consistent, high-quality datasets. Simply merging existing datasets often introduces inconsistencies in resolution and background quality. To solve this, the authors released URIR-8K, a comprehensive benchmark containing 8,000 high-resolution image pairs.

The dataset is built upon backgrounds from the BDD100K driving dataset, ensuring relevance for robotics and automotive applications. It covers four distinct categories:

  1. Rain Streaks (RS): Standard linear streaks.
  2. Raindrops (RD): Large drops adhering to the camera lens.
  3. Rain Accumulation (RA): Heavy rain creating a fog-like scattering effect.
  4. Nighttime Rain (NR): Rain patterns under complex artificial lighting.

Notably, URIR-8K includes object detection labels, enabling researchers to evaluate the impact of restoration on downstream perception tasks.

Multi-Scale Deraining Mamba (MSDM)

To tackle the multi-scale nature of rain, the authors proposed the Multi-Scale Deraining Mamba (MSDM). This architecture leverages State Space Models (SSMs), specifically the Mamba framework, which offers linear complexity and a global receptive field, making it more efficient than traditional Transformers.

The MSDM architecture utilizes multi-scale processing and frequency-guided modules to handle diverse rain patterns.

The MSDM incorporates several key technical innovations:

  • Multi-Scale 2D Scanning: A piecewise scanning strategy that adapts to different image scales, using more complex geometric transformations for larger scales to capture fine details.
  • Frequency-Guided Convolutional Module (FGCM): Operates in the frequency domain to effectively separate high-frequency rain components from low-frequency background structures.
  • Multi-Scale Gated Fusion (MGFM): A dynamic fusion mechanism that uses gating units to selectively integrate features across different scales.

Performance and Efficiency

The MSDM model demonstrates exceptional versatility, often described as a "pentagonal warrior" for its balanced performance across all rain types and downstream detection tasks.

Radar chart comparing MSDM (Ours) against state-of-the-art methods across five metrics: PSNR for four rain types and mAP50 for object detection.

On the URIR-8K benchmark, MSDM achieved an average PSNR of 39.35 dB, significantly outperforming previous state-of-the-art models like IDT. Beyond pixel-level restoration, the model nearly doubled object detection accuracy (mAP50) compared to raw rainy images, increasing it from 0.200 to 0.402.

Crucially, these gains were achieved with high efficiency. MSDM uses only 11.8M parameters and 99.2 GFLOPs, representing a reduction of over 60% in parameters compared to heavy-duty models like Restormer, while delivering superior results.

Conclusion and Future Directions

The introduction of URIR-8K and the MSDM architecture marks a significant step toward robust, real-world image restoration. By moving away from specialized "expert" models toward a universal baseline, the research provides a more practical path for deploying vision systems in adverse weather. Future work may focus on bridging the remaining synthetic-to-real gap, ensuring temporal consistency in video sequences, and extending the framework to handle other extreme weather conditions like snow or mixed precipitation.

Summary updated

BibTeX
@inproceedings{yan2025towards,
  author = {Hujie Yan},
  title = {{Towards Universal Rainy Image Restoration: Benchmark and Baseline}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i9.32985},
}
Figure