Abstract
Adverse weather conditions such as rain, snow, and haze significantly degrade image quality, posing challenges for outdoor computer vision systems. We introduce CMAWRNet, a unified neural architecture designed to remove multiple weather effects simultaneously. By integrating quaternion neural networks with a novel texture-structure decomposition strategy, CMAWRNet achieves state-of-the-art restoration quality. Notably, the model maintains high color fidelity and structural integrity while requiring significantly fewer parameters and offering faster inference times than existing real-valued models.
Introduction
Real-world computer vision systems, particularly those used in autonomous vehicles and outdoor surveillance, must operate reliably under diverse environmental conditions. However, adverse weather often introduces complex degradations that "specialist" models—trained for a single task like dehazing or deraining—struggle to handle. Universal models have emerged to address this, but they frequently suffer from over-smoothing or fail to preserve natural color correlations because they treat RGB channels as independent entities.
CMAWRNet addresses these limitations by leveraging the mathematical properties of quaternions to represent color information holistically and employing a "divide and conquer" approach to separate different types of weather-induced noise.
Methodology
The core of CMAWRNet is a multi-stage pipeline that decomposes, cleans, and fuses image components to restore clarity.
1. Quaternion Representation
Unlike standard networks that use real-valued neurons, CMAWRNet utilizes quaternions (four-dimensional numbers). An RGB pixel is represented as a single quaternion entity ($a + bi + cj + dk$), where the imaginary parts correspond to the color channels. This representation allows the network to capture and preserve the complex inter-channel correlations that are often lost in traditional processing, leading to more accurate color restoration and avoiding unnatural tints.
2. Texture-Structure Decomposition (DNet)
Inspired by Retinex theory, the DNet module splits the degraded input image into two distinct components:
- Structure: Captures smooth, low-frequency information, which is primarily affected by global degradations like haze and fog.
- Texture: Captures high-frequency details, where local artifacts like rain streaks and snowflakes are typically concentrated.
3. Lightweight Transformer Cleaning (TNet)
The decomposed components are processed by parallel TNet modules. These quaternion-based transformers are designed to be highly efficient, achieving similar results to real-valued layers with roughly 1/4 the parameters. By targeting texture and structure separately, the model can remove specific artifacts without blurring the underlying scene details.
4. Attentive Fusion (FNet)
The FNet module recombines the cleaned components. It employs an attentive fusion mechanism that also performs low-light correction, ensuring that the final restored image is both clear and appropriately illuminated.
Experimental Evaluation and Results
CMAWRNet was evaluated against several state-of-the-art (SOTA) models, including TransWeather and MPRNet, across multiple benchmark datasets such as RainDrop, Snow100K, and Outdoor-Rain.
Quantitative Performance
The model demonstrated superior performance in mixed-weather scenarios. On the "Rain+Fog" test set, CMAWRNet achieved a PSNR of 30.02 dB, outperforming the next best model by nearly 2 dB. This improvement is attributed to the model's ability to handle global and local degradations simultaneously through its decomposition strategy.
Efficiency and Speed
Efficiency is a critical factor for real-time applications. CMAWRNet processes a 1080p image in just 24.3 ms on an NVIDIA A100 GPU—nearly twice as fast as some competing universal models. This speed, combined with its reduced parameter count, makes it a strong candidate for deployment in resource-constrained environments like autonomous vehicle hardware.
Downstream Task Impact
To validate the practical utility of the restoration, the authors tested object detection performance on the real-world DAWN dataset. Using CMAWRNet as a pre-processor significantly improved the Mean Average Precision (mAP), particularly for small objects that are often completely obscured by heavy weather noise.
Conclusion
CMAWRNet represents a significant step forward in universal weather removal. By combining the mathematical elegance of quaternion neural networks with a robust decomposition-fusion framework, it provides a lightweight yet powerful solution for restoring images in adverse conditions. Future work will focus on optimizing the architecture for high-frame-rate performance on lower-power edge devices and extending the decomposition approach to other restoration tasks like deblurring.
Summary updated