Introduction
In real-world applications such as autonomous driving and outdoor surveillance, vision systems must operate under diverse and dynamic atmospheric conditions. While specialized models exist for individual tasks like dehazing, deraining, or desnowing, deploying a separate model for every weather condition is computationally expensive and impractical. Conversely, training a single model on all weather types simultaneously (joint training) requires massive, pre-collected datasets that may not be available in evolving environments.
This paper addresses the challenge of Continual All-in-One Adverse Weather Removal. The goal is to develop a system that learns to handle new weather conditions sequentially without "forgetting" how to process previous ones—a phenomenon known as catastrophic forgetting. The authors propose a novel framework utilizing a unified network structure and a Knowledge Replay (KR) mechanism enhanced by Principal Component Projection (PCP).
The Problem of Catastrophic Forgetting
Standard deep learning models are typically "static." When a model trained for dehazing is fine-tuned on a new task, such as deraining, the weights are optimized for the new objective, leading to a sharp decline in performance on the original task. In image restoration, this is particularly problematic because the mapping from a degraded image to a clean one is a "many-to-one" problem; different weather artifacts require different restoration logic, yet they all aim for the same clean underlying structure.
Proposed Methodology
1. Unified Network Structure
The framework utilizes a unified architecture based on the Feature Fusion Attention Network (FFA-Net). Unlike multi-task models that require a "task ID" to select specific branches during inference, this unified structure employs a shared feature extractor and projector head. This allows the model to handle any weather type blindly during deployment.
2. Knowledge Replay (KR) with Teacher-Student Distillation
To prevent forgetting, the authors implement a Knowledge Replay strategy. When the model (the "student") encounters a new weather task, it is guided by the previous version of itself (the "teacher"). The teacher provides supervision for old tasks using a small memory buffer of 500 samples. Notably, this method does not require ground-truth clean images for the replay samples; the teacher's output serves as the reference, reducing storage requirements by approximately 50% compared to traditional rehearsal methods.
3. Principal Component Projection (PCP)
The core innovation is the Principal Component Projection (PCP) module used for mid-level feature distillation. The authors argue that simply distilling the final output is insufficient to preserve complex structural knowledge. PCP performs distillation on mid-level features by mimicking Principal Component Analysis (PCA). By projecting features into a lower-dimensional space that captures essential structural information while filtering out task-specific noise, the model maintains a balance between "plasticity" (learning new weather) and "stability" (retaining old knowledge).
Experimental Results
Quantitative Performance
The framework was evaluated on three major synthetic datasets: OTS (dehazing), Rain100H (deraining), and Snow100K (desnowing).
- Performance Stability: The proposed method achieved an average PSNR of 30.70 dB across all tasks. This is remarkably close to the "Joint Training" upper bound of 31.27 dB, where all data is available at once.
- Resistance to Forgetting: After learning all three tasks, the performance drop on the initial dehazing task was only 0.41 dB. In contrast, standard fine-tuning saw performance plummet to 20.08 dB, effectively losing all prior utility.
Downstream Utility
To demonstrate real-world efficacy, the restored images were used as input for an object detection model on the FoggyCityscape dataset. The restoration process improved the Mean Average Precision (mAP) from 26.36% to 29.30%, confirming that the "all-in-one" approach significantly benefits machine vision tasks in adverse conditions.
Conclusion
This research presents the first successful framework for continual all-in-one adverse weather removal. By combining a unified network with a memory-efficient Knowledge Replay mechanism and Principal Component Projection, the authors demonstrate that models can learn incrementally without the need for massive data storage or catastrophic performance trade-offs. Future work may extend this framework to other restoration tasks, such as low-light enhancement and motion blur removal, further bridging the gap between synthetic training and complex real-world environments.
Summary updated