RAM++: Robust Representation Learning via Adaptive Mask for All-in-One Image Restoration

Zilong Zhang, Chujie Qin, Chunle Guo, Yong Zhang, Chao Xue, Ming-Ming Cheng, Chongyi LiarXiv 2025 · DOI · arXiv

Abstract

RAM++ is a novel "all-in-one" image restoration framework designed to address multiple types of degradation—including noise, rain, haze, and blur—within a single model. Unlike traditional degradation-oriented approaches that attempt to classify and subtract specific noise patterns, RAM++ shifts the focus toward learning the intrinsic content of natural images. By employing adaptive masked pre-training and DINOv2-guided fine-tuning, the framework achieves state-of-the-art, balanced performance across both seen and unseen degradations, demonstrating superior robustness in extreme scenarios.

Introduction

In real-world applications, images are frequently corrupted by environmental or hardware factors such as rain, low light, motion blur, or sensor noise. While specialized models exist for individual tasks, "all-in-one" models aim to handle multiple degradations simultaneously. However, current all-in-one models are primarily degradation-oriented, attempting to map specific corruptions into distinct feature spaces. This often leads to three critical issues: insufficient generative capability in extreme scenarios, unbalanced performance across different tasks (where improving one task degrades another), and weak generalization to unseen or mixed degradations.

RAM++ addresses these flaws by proposing that models should learn what a clean image looks like rather than what degradation looks like. This shift from degradation-oriented to content-oriented learning allows for more robust and generalized image restoration.

Conceptual shift: RAM++ focuses on semantic content (e.g., zebra, grass) rather than just identifying degradation types (e.g., noise, haze).

Methods

RAM++ utilizes a two-stage framework consisting of pre-training and fine-tuning, centered on three core innovations:

  1. Adaptive Semantic-Aware Mask (AdaSAM) Pre-training: Standard Masked Image Modeling (MIM) often uses large patches that destroy local textures essential for restoration. AdaSAM introduces adaptive, pixel-level masking focused on semantically rich and highly textured regions. This forces the model to reconstruct complex structures, building strong generative image priors.
  2. Mask Attribute Conductance (MAC): To bridge the distribution gap between masked images (pre-training) and full images (inference), MAC uses a gradient-based analysis to identify critical network layers. By selectively fine-tuning only the top ~30% of these layers, the model adapts to full images while preserving the powerful priors learned during pre-training.
  3. Robust Feature Regularization (RFR): During fine-tuning, RAM++ integrates representations from DINOv2, a pre-trained vision foundation model. DINOv2 features are highly semantic and naturally invariant to low-level degradations, helping RAM++ maintain structural coherence even under extreme corruption.

Key Results and Evaluation

The authors evaluated RAM++ across multiple benchmarks, including 3-task and 7-task settings, as well as Out-of-Distribution (OOD) and extreme degradation scenarios (e.g., CDD-11, UIEB).

  • State-of-the-Art Performance: In a comprehensive 7-task setting, RAM++ improved average PSNR by 0.70 dB over existing best methods.
  • Balanced Robustness: As shown in the radar chart, RAM++ maintains high performance across diverse conditions without the typical trade-offs seen in previous models. It achieved a massive 1.84 dB PSNR gain over the Restormer baseline when tested on completely unseen noise types (e.g., speckle noise).
  • Efficiency: Using MAC to fine-tune just 30% of the network, RAM++ still outperformed fully trained competitive models, highlighting the effectiveness of its pre-trained representation.
Radar chart showing the balanced and robust performance of RAM++ across diverse degradation conditions compared to previous methods.

Conclusion

RAM++ demonstrates that prioritizing intrinsic image content over specific degradation patterns leads to more stable and generalized image restoration. By combining adaptive masking with foundation model guidance, it sets a new benchmark for all-in-one restoration. Future work will focus on optimizing data-mixing strategies and extending the framework to video restoration to ensure temporal consistency across frames.

Summary updated

Abstract

This work presents Robust Representation Learning via Adaptive Mask (RAM++), a two-stage framework for all-in-one image restoration. RAM++ integrates high-level semantic understanding with low-level texture generation to achieve content-oriented robust restoration. It addresses the limitations of existing degradation-oriented methods in extreme scenarios (e.g., degradations strongly coupled with image structures). RAM++ also mitigates common challenges such as unbalanced performance across tasks, overfitting to seen degradations, and weak generalization to unseen ones through three key designs: 1) Adaptive Semantic-Aware Mask (AdaSAM): a pretraining strategy that applies pixel-level masks to semantically rich and textured regions. This design enables the network to learn both generative priors and image content priors from various degradations. 2) Mask Attribute Conductance (MAC): a selective fine-tuning strategy that adjusts the layers with higher contributions to bridge the integrity gap between masked pretraining and full-image fine-tuning while retaining learned priors. 3) Robust Feature Regularization (RFR): a strategy that leverages DINOv2's semantically consistent and degradation-invariant representations, together with efficient feature fusion, to achieve faithful and semantically coherent restoration. With these designs, RAM++ achieves robust, well-balanced, and state-of-the-art performance across seen, unseen, extreme, and mixed degradations. Our code and model will be released at https://github.com/DragonisCV/RAM
BibTeX
@misc{zhang2025ram,
  author = {Zilong Zhang and Chujie Qin and Chunle Guo and Yong Zhang and Chao Xue and Ming-Ming Cheng and Chongyi Li},
  title = {{RAM++: Robust Representation Learning via Adaptive Mask for All-in-One Image Restoration}},
  year = {2025},
  eprint = {2509.12039},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.12039},
}
Figure