Abstract
Self-supervised pretraining has revolutionized high-level vision, yet low-level vision tasks like dehazing and deraining have lacked a comparable foundation paradigm. We present DegAE (Degradation Autoencoder), a novel framework that treats pretraining as a degradation transfer problem. By forcing a model to disentangle image content from degradation and re-synthesize corrupted versions, DegAE learns general representations that yield significant performance gains in data-scarce scenarios. Our method achieves up to a 6.88dB PSNR improvement in image dehazing, demonstrating its efficacy for "high-cost" restoration tasks.
Introduction
In high-level vision tasks such as classification and detection, Masked Image Modeling (MIM) has become the gold standard for self-supervised pretraining. However, applying MIM to low-level vision—tasks focused on restoring fine-grained image quality—presents two major hurdles. First, the Masking Dilemma: MIM hides image patches to encourage semantic learning, but this process destroys the high-frequency spatial information (edges and textures) essential for restoration. Second, the "Low-Cost" Trap: Many tasks like denoising use synthetic data that is easily generated, meaning pretraining often yields marginal gains because downstream data is already effectively infinite.
The real challenge lies in "high-cost" tasks like dehazing or motion deblurring, where collecting real-world paired data is difficult and expensive. DegAE addresses this by providing a pretraining scheme specifically optimized for these data-starved scenarios.
The DegAE Framework
DegAE shifts the focus from pixel masking to degradation transfer. Instead of reconstructing missing pixels, the model learns to manipulate the degradation state of an image while preserving its underlying structure.
The Pretext Task
During pretraining, the model receives an input image with one type of degradation ($D_1$) and a reference image with a different degradation ($D_2$). The objective is to produce an output that retains the content of the input image but adopts the degradation characteristics of the reference image.
To succeed at this task, the encoder must learn to extract a "clean" representation of the content, effectively ignoring specific noise or blur. Simultaneously, the system encodes the degradation characteristics of the reference. This disentanglement allows the model to develop a deep "prior" for what natural, clean images look like versus how various corruptions manifest.
Architecture and Training
The framework utilizes a Transformer-based encoder (compatible with backbones like SwinIR, Uformer, or Restormer) and a CNN-based decoder. The decoder is used exclusively during pretraining to facilitate the degradation transfer. For downstream tasks, the pretrained encoder is retained, while the decoder is replaced with a simple, task-specific head. Training is guided by a combination of content reconstruction loss, perceptual loss (VGG-based), adversarial loss, and a novel embedding loss to ensure the output degradation accurately matches the reference.
Experimental Results
DegAE was evaluated across several challenging downstream tasks using large-scale datasets like ImageNet and DF2K for pretraining. The results demonstrate that DegAE provides substantial ROI, particularly for high-cost tasks:
- Image Dehazing: Using a SwinIR backbone, DegAE achieved a +6.88dB PSNR improvement over training from scratch on the SOTS-indoor dataset. Uformer saw a +3.22dB boost.
- Image Deraining: On the Rain100L benchmark, SwinIR with DegAE improved by 1.27dB, while Uformer improved by 0.54dB.
- Motion Deblurring: On the GoPro dataset, the method provided gains of 0.47dB for SwinIR and 0.43dB for Restormer.
Qualitatively, models pretrained with DegAE produce fewer artifacts and more natural textures. While scratch-trained models often leave residual haze or "box" artifacts, DegAE-enhanced models preserve fine details and color fidelity more effectively.
Conclusion
DegAE establishes a new paradigm for low-level vision pretraining by prioritizing degradation-content disentanglement over simple reconstruction. By learning to generate specific degradations on new content, the model implicitly learns the inverse—how to remove them. This approach offers a powerful solution for improving performance in complex restoration tasks where paired training data is a primary bottleneck.
Summary updated