This paper, titled "Always Clear Days: Degradation Type and Severity Aware All-In-One Adverse Weather Removal," introduces a novel deep learning framework designed to clean up images obscured by various weather conditions like rain, snow, and haze using a single, unified model.
TL;DR
The authors propose UtilityIR, an "all-in-one" restoration model that explicitly accounts for both the type and severity of weather degradation. By using a novel ranking loss and adaptive normalization, it outperforms state-of-the-art models with fewer parameters and allows users to manually "tune" the level of restoration.
What Problem Does This Paper Address?
Most computer vision algorithms (like those in self-driving cars) are trained on clear images and fail miserably in adverse weather. While researchers have built specific models to remove rain or snow individually, "all-in-one" models—which handle multiple weather types at once—are the holy grail for practical deployment.
The authors identify a major flaw in existing all-in-one models: they focus on weather types (e.g., "is this rain or snow?") but ignore weather severity (e.g., "is this a light drizzle or a torrential downpour?"). Ignoring severity leads to "intra-domain gaps," where a model might under-restore a heavy storm or over-process a light mist, creating artifacts. Furthermore, weather often varies spatially—raindrops are local spots, while fog is a global haze—requiring a model that can handle both local and global distortions simultaneously.
Methods
The core of the proposed UtilityIR (Utility Image Restoration) is the Degradation Information Encoder (DIE), which extracts two specific signals from a degraded image:
- Weather Type Extraction: Uses Contrastive Loss (CL) to ensure that features from the same weather type (e.g., all snow images) cluster together in the model's "mind," while different types are pushed far apart.
- Weather Severity Estimation: Introduces a new Marginal Quality Ranking Loss (MQRL). Unlike standard ranking losses that only care if one image is "worse" than another, MQRL forces the model to learn the interval or degree of difference in quality. This allows the model to accurately gauge exactly how much restoration is needed.
To apply this information, the authors use two clever architectural tricks:
- LG-AdaIN (Local-Global Adaptive Instance Normalization): This layer uses the weather type to adjust pixels locally (good for raindrops) and uses the severity to adjust the image globally (good for haze).
- DGCA (Degradation-guided Cross Attention): This uses the extracted weather information as a "query" to help the transformer-style layers focus on the most degraded parts of the image.
Datasets
The model was evaluated using two primary experimental setups:
- Setting 1 (All-Weather): A combination of the Raindrop dataset, Outdoor-Rain (rain + fog), and Snow100K-L.
- Setting 2 (Standard Benchmarks): Includes Rain1400 (rain streaks), RESIDE OTS (haze), and Snow100K-L.
- Real-World Data: The authors also tested the model on various unlabelled real-world images to prove its generalization capability.
Key Insights and Ideas
- Severity is a Spectrum: The "aha moment" here is treating weather not as a set of discrete categories, but as a continuous spectrum of intensity. By learning the variance in degradation, the model becomes much more flexible.
- User-Controllable Restoration: Because the model represents severity as a vector in a latent space, the authors found they could "nudge" this vector. This allows a user to manually modulate the restoration level—effectively a "de-weathering" slider.
- Zero-Shot Combination: Surprisingly, even though the model was trained on single weather types (just rain or just snow), it can handle unseen combinations (like rain + snow + fog) by iteratively processing the image.
Key Results
- Quantitative Superiority: UtilityIR outperformed previous leaders (like Transweather and Restormer) by significant margins. For example, it achieved a +2.65 dB PSNR improvement on the challenging rain+fog dataset.
- Efficiency: Despite the better performance, UtilityIR uses only 26.3 million parameters, making it significantly lighter than competitors like WeatherDiff (83M) or All-in-one (44M).
- Qualitative Quality: Visually, the model avoids common pitfalls like "black stain" artifacts left behind after raindrop removal or the "over-smoothing" of textures common in haze removal.
Limitations and Future Work
The authors are transparent about a few hurdles:
- Parameter Sensitivity: The new MQRL loss requires careful tuning of its "margin" parameter to work effectively.
- Computational Cost: While the parameter count is low, the use of cross-attention mechanisms still demands high computational power during inference.
- Severe Occlusion: In cases where weather completely blocks high-frequency details (like a thick snowflake covering a small sign), the model—like all current SOTA methods—struggles to "hallucinate" the missing information perfectly, sometimes resulting in slight blurring.
Future work aims to make the model more robust to completely "unseen" weather types encountered in the wild.
Summary updated