Introduction
Image restoration is a fundamental task in computer vision, traditionally addressed through specialized models tailored to specific degradations such as noise, blur, or haze. While effective, this "one-model-per-problem" approach is computationally expensive and difficult to deploy in real-world scenarios where multiple degradations may coexist or vary. The emergence of "all-in-one" restoration aims to consolidate these capabilities into a single universal model.
However, all-in-one models face a significant hurdle known as task interference. Because different restoration tasks—such as removing rain streaks versus brightening a low-light scene—require distinct mathematical operations, the gradient updates during training often conflict. Improving performance on one degradation can inadvertently lead to performance drops in others. This paper introduces DFPIR, a novel framework that utilizes degradation-aware feature perturbation to dynamically reorganize features, effectively mitigating task interference without the need for massive parameter expansion.
Methodology: The DFP Strategy
The core innovation of DFPIR lies in the Degradation-Aware Feature Perturbation (DFP) strategy, integrated into a backbone based on the Restormer architecture. The system utilizes textual prompts (e.g., "a rainy image") encoded via a pre-trained CLIP model to guide the restoration process. These prompts inform the Degradation-Guided Perturbation Block (DGPB), which consists of two primary modules designed to "clean" and reorganize features within the skip-connections between the encoder and decoder.
1. Channel-wise Perturbation (DGCPM)
Unlike standard attention mechanisms that simply re-weight channels, the Degradation-Guided Channel Perturbation Module (DGCPM) employs a "hard routing" approach. It first expands the feature space to double the channel count, providing sufficient high-dimensional "room" to prevent information loss. It then shuffles the channel order based on the specific degradation type. This reordering aligns the feature space with the shared parameters of the decoder, ensuring that the network processes different types of damage through distinct paths in the high-dimensional space.
2. Attention-wise Perturbation (CAAPM)
To further reduce mutual influence between tasks, the Channel-wise Attention-based Attention Perturbation Module (CAAPM) applies a "top-K" mask to the attention maps. By selectively masking out a portion of the attention for each specific degradation, the model focuses exclusively on the most relevant features for the current task. This selective attention prevents the features associated with one degradation (like haze) from polluting the reconstruction process of another (like noise).
Experimental Setup and Results
The researchers evaluated DFPIR against a suite of standard benchmarks across five major restoration tasks: denoising (CBSD68, Urban100), dehazing (SOTS), deraining (Rain100L), deblurring (GoPro), and low-light enhancement (LOL-v1).
Quantitative Performance
DFPIR demonstrated superior performance in both 3-task (Haze, Rain, Noise) and 5-task (adding Blur and Low-light) configurations:
- 3-Task Setting: DFPIR achieved an average Peak Signal-to-Noise Ratio (PSNR) of 32.88 dB, outperforming the previous state-of-the-art, InstructIR, by 0.45 dB.
- 5-Task Setting: In the more challenging five-degradation scenario, DFPIR reached 30.64 dB, representing a substantial 1.09 dB improvement over InstructIR.
Qualitative Analysis
Visual comparisons indicate that DFPIR is particularly adept at preserving fine textures and structural details. While competing all-in-one models often smudge intricate patterns—such as the fine grill of a vehicle or individual raindrops—DFPIR maintains high fidelity. t-SNE visualizations further confirm that the perturbation strategy successfully clusters features from different tasks into distinct, tightly organized groups in the feature space, validating the effectiveness of the "shuffling as routing" philosophy.
Discussion and Insights
The success of DFPIR highlights two critical insights for all-in-one restoration:
- Shuffling vs. Weighting: While soft weighting is common in attention mechanisms, "hard" channel shuffling acts as a more effective routing mechanism for multi-task learning. It preserves the inherent content of the image while forcing the network to adopt task-specific processing paths.
- Dimensionality Matters: The authors found that shuffling in low-dimensional space is too disruptive for convergence. Doubling the channel count before perturbation is essential for maintaining the integrity of the image data while allowing for reorganization.
Conclusion and Future Work
DFPIR sets a new benchmark for all-in-one image restoration by addressing task interference through intelligent feature perturbation. While the model currently relies on accurate degradation prompts, it offers a scalable and efficient alternative to maintaining multiple specialized networks. Future research may explore extending this perturbation philosophy to video restoration or blind restoration scenarios where the degradation type is not explicitly known.
Summary updated