Abstract
DINO-IR is a novel multi-task image restoration framework that leverages robust, degradation-insensitive features from the DINOv2 foundation model. By employing these semantic "anchors" to guide the restoration process, the model achieves state-of-the-art performance across denoising, deblurring, deraining, and dehazing. Crucially, DINO-IR maintains significantly higher stability than previous methods as the number of tasks increases, effectively mitigating the performance drop typically associated with multi-task interference.
Introduction
Image restoration is a fundamental computer vision challenge aimed at recovering high-quality images from versions corrupted by noise, blur, rain, or haze. While specialized models for individual tasks have achieved high performance, there is a growing shift toward unified multi-task models for improved efficiency. However, multi-tasking often introduces "task interference," where the optimization goals of different degradations conflict. For example, denoising seeks to smooth high-frequency noise, while deblurring aims to sharpen high-frequency details. As more tasks are added to a single model, performance typically degrades. DINO-IR addresses this by utilizing degradation-agnostic semantic representations as stable anchors to guide the restoration process.
Methodology
The DINO-IR framework integrates the robust feature extraction capabilities of DINOv2—a large-scale self-supervised vision transformer—into a restoration backbone, specifically Restormer. The architecture is designed around three core components:
- Pixel-Semantic Fusion (PSF) Module: This module exploits the hierarchical nature of DINOv2. Shallow layers provide fine-grained spatial details, while deeper layers offer high-level semantic context. A gating network dynamically weights these features, selecting the most relevant information for the specific restoration task at hand.
- DINO-Restore (D-R) Adaption and Fusion: This component bridges the gap between the foundation model and the restoration pipeline. It adapts DINO features to the required dimensions and utilizes a self-attention mechanism to fuse them effectively into the main restoration flow.
- DINO Perception Contrastive (DPC) Loss: During training, this loss function encourages the restored image's features to align with the clean ground truth while distancing them from the degraded input in the feature space.
Key Insights: DINOv2 as a Robust Anchor
The central innovation of DINO-IR is the use of DINOv2 features as a "robust anchor." Empirical analysis reveals that these high-level features remain remarkably consistent regardless of whether an image is clean, rainy, or blurry. This stability allows the model to maintain structural integrity during restoration, even under heavy corruption. Furthermore, by incorporating semantic guidance, the model moves beyond local pixel-level operations to understand underlying object shapes and textures, leading to more accurate and realistic reconstructions than models relying solely on local neighborhoods.
Experimental Results and Stability
The model was evaluated on a comprehensive suite of standard benchmarks, including DID and Rain100L for deraining, SOTS for dehazing, and Urban100 for denoising. DINO-IR consistently outperformed existing multi-task leaders such as AirNet and PromptIR, showing an average improvement of 0.13 dB to 1 dB in PSNR across various tasks. A particularly notable milestone was achieved in dehazing, where the model gained up to 3.41 dB over previous multi-task methods.
A key strength of DINO-IR is its "enhanced stability." While traditional multi-task models see a sharp decline in performance as the number of tasks grows, DINO-IR exhibits much greater resilience. This is visualized in the comparison between the base Restormer and the DINO-guided version, where the performance gap widens as more tasks are added.
Conclusion and Future Work
DINO-IR represents a significant advancement in robust, general-purpose image restoration. By grounding the restoration process in stable semantic features, it overcomes the traditional pitfalls of multi-task interference. While the model shows superior quality and generalization, the use of a large foundation model like DINOv2 introduces computational overhead. Future research will likely explore methods to reduce this cost and evaluate the framework's performance on complex, "in-the-wild" camera artifacts that go beyond synthetic degradations.
Summary updated