TL;DR
UniRain is a unified image deraining framework that simultaneously handles rain streaks and raindrops across both daytime and nighttime conditions. By combining a novel Retrieval-Augmented Generation (RAG) dataset distillation pipeline with a multi-objective reweighted optimization strategy, it overcomes data quality disparities and task imbalances to deliver state-of-the-art restoration performance with high computational efficiency.
What Problem Does This Paper Address?
Most deep learning-based image deraining methods are hyper-specialized. A model trained to remove daytime rain streaks will often fail miserably when faced with nighttime raindrops, where low illumination and complex light reflections alter the degradation pattern.
To build an "all-in-one" universal derainer, the intuitive approach is to simply merge all public datasets. However, the authors identify two major roadblocks to this strategy:
- Data Quality Disparity: Public datasets vary wildly in realism and background quality. Directly mixing them introduces noisy, inaccurate supervisory signals that hinder model convergence.
- Task Imbalance: Different rain types present different learning difficulties. During joint training, a standard loss function causes the model to overfit to easier tasks (like nighttime streaks) while neglecting more complex ones (like daytime raindrops).
Methods
The authors introduce UniRain, a framework consisting of three core innovations designed to solve the dataset and optimization challenges of unified deraining:
1. RAG-based Dataset Distillation
Instead of using raw, noisy datasets, the authors design an offline dataset distillation pipeline. For every training image, the system uses a hierarchical similarity matching process (textual captions via BLIP, visual features via CLIP, and structural similarity via SSIM) to retrieve the most relevant real-world rainy images from a curated database. An ensemble of Vision-Language Models (VLMs)—including InternVL2.5, LLaVA-NeXT, and MobileVLM—then evaluates the query image against these real-world references to filter out low-fidelity or unrealistic training samples.
2. Asymmetric Mixture-of-Experts (MoE) Architecture
To handle diverse rain types efficiently, UniRain adopts an asymmetric MoE structure:
- Soft-MoE Encoder: Uses continuous routing weights to softly combine expert representations, ensuring that diverse degradation cues are collaboratively preserved.
- Hard-MoE Decoder: Employs discrete Top-$k$ routing to activate only the most relevant experts, focusing computational power on sharp, fine-grained texture reconstruction.
3. Multi-objective Reweighted Optimization
To prevent the model from favoring easier rain types, the authors introduce a dynamic loss-reweighting mechanism. By tracking the Type Balance Score (TBS) (how fast a task is converging) and the Type Stability Score (TSS) (how consistently it converges) within a sliding temporal window, the optimizer dynamically boosts the loss weights of lagging or unstable tasks, ensuring synchronized learning.
Datasets
- RainRAG (New): The authors curated this high-quality dataset by running their RAG distillation pipeline over more than 2 million public synthetic and real-world image pairs. The pipeline retained only the top ~2.6% highest-quality samples, resulting in 52,869 training pairs split into four categories: daytime rain streaks (DRS), daytime raindrops (DRD), nighttime rain streaks (NRS), and nighttime raindrops (NRD).
- Evaluation Benchmarks: The model was tested on the RainRAG test set, as well as three real-world public benchmarks: RealRain-1k, RainDS-real, and WeatherBench.
Key Insights and Ideas
- Quality Over Quantity: A key takeaway from the ablation studies is that training on a massive, unfiltered dataset (~2,000k images) yields worse results than training on a clean, distilled subset (~52k images). In low-level vision, data realism and consistency matter more than sheer volume.
- Stability Matters as Much as Speed: Traditional multi-task optimization often balances tasks based solely on their instantaneous loss or convergence speed. UniRain demonstrates that incorporating a historical stability metric prevents the optimizer from over-allocating weights to tasks that are wildly oscillating or diverging.
- Asymmetry is Efficient: Combining soft routing in the encoder with hard routing in the decoder strikes an ideal balance between broad feature exploration and sharp, localized feature exploitation.
Key Results
- Quantitative Performance: On the RainRAG dataset, UniRain achieved an average PSNR of 28.93 dB, outperforming the previous state-of-the-art universal method, URIR, by 1.02 dB. On real-world benchmarks, it achieved a substantial average improvement of 1.73 dB PSNR over URIR.
- Computational Efficiency: Despite its superior performance, UniRain is highly efficient, requiring only 126.54 GFLOPs and 24.38M parameters, making it significantly lighter than specialized architectures like DRSformer (220.38 GFLOPs, 33.62M parameters).
- Downstream Generalization: The model showed robust zero-shot generalization across diverse unseen operational domains, including autonomous driving, unmanned aerial vehicles (UAVs), and maritime environments.
Limitations and Future Work
While the authors successfully extended UniRain to an "all-in-one" weather restoration task (handling rain, snow, and haze simultaneously), the framework treats different weather conditions as independent optimization objectives. In the real world, weather conditions often overlap (e.g., rainy and foggy conditions occurring together). Future work could explore modeling the physical and feature-level interactions between concurrent weather degradations rather than treating them as isolated tasks.
Summary updated