Zero-Shot Image Super-Resolution with Depth Guided Internal Degradation Learning

Xi Cheng, Zhenyong Fu, Jian YangECCV 2020 · DOI

Introduction

Most modern image super-resolution (SR) models rely on supervised learning, requiring massive datasets of high-resolution (HR) and low-resolution (LR) pairs. Typically, these pairs are generated using simplistic bicubic downscaling. However, real-world degradation is far more complex, involving sensor noise, motion blur, and lens optics. When models trained on "clean" data encounter real-world images, they often produce artifacts or blurry results.

This paper introduces DGDML-SR (Depth Guided Degradation Model Learning-based SR), a zero-shot method that learns to restore images using only the input image itself. By leveraging depth information as a natural proxy for scale, the model learns an image-specific degradation process through a "bi-cycle" training strategy.

Methodology: Depth as a Scale Supervisor

The core innovation of DGDML-SR is the use of depth information to identify training examples within a single image. The authors observe a "depth-guided self-similarity prior": objects at different distances appear at different scales within the same scene.

Depth-guided data generation: patches of similar objects at different depths (Close, Far, Farther) provide natural multi-scale training pairs.

Dual-Network Architecture and Bi-cycle Training

The framework utilizes two primary networks:

  1. Degradation Simulation Network (DSN): Learns the specific degradation process of the image (HR to LR).
  2. Super-Resolution Network (SRN): Learns the inverse restoration process (LR to HR).

Because the extracted patches are "unpaired" (they represent similar objects but not identical pixel regions), the authors employ a bi-cycle training strategy. Similar to CycleGAN, one cycle maps LR → HR → LR, while the other maps HR → LR → HR. This ensures consistency and prevents the model from collapsing or "cheating" during the internal learning process.

Key Results and Perceptual Quality

The model was evaluated on datasets including NYU Depth V2, B3DO, and Urban100. Evaluation focused on perceptual metrics (NIQE and PI), where lower scores indicate superior visual quality, as traditional metrics like PSNR often fail to capture the sharpness required for real-world applications.

Qualitative comparison: DGDML-SR recovers sharper edges and finer textures compared to supervised (RCAN) and other zero-shot (ZSSR, KernelGAN) methods.

On the NYU Depth dataset (×2 scale), DGDML-SR achieved a NIQE score of 4.824, significantly outperforming the supervised state-of-the-art RCAN (5.868) and the zero-shot ZSSR (5.753). Visual comparisons demonstrate that DGDML-SR effectively recovers fine textures and sharp edges without the over-smoothing or ringing artifacts common in other methods.

Conclusion and Future Work

DGDML-SR demonstrates that depth maps can serve as effective "scale maps" for self-supervised SR. By learning the degradation and restoration simultaneously in a unified loop, the method adapts to the unique characteristics of a specific camera and scene. While the method currently depends on the availability of depth maps and requires per-image training time, it sets a new benchmark for perceptual quality in zero-shot super-resolution. Future work may focus on optimizing patch selection to further refine the training signal in scenes with unique, non-recurring textures.

Summary updated

BibTeX
@inproceedings{cheng2020zero,
  author = {Xi Cheng and Zhenyong Fu and Jian Yang},
  title = {{Zero-Shot Image Super-Resolution with Depth Guided Internal Degradation Learning}},
  booktitle = {European Conference on Computer Vision},
  year = {2020},
  doi = {10.1007/978-3-030-58520-4\_16},
}
Figure