Unsupervised Single Image Super-Resolution Network (USISResNet) for Real-World Data Using Generative Adversarial Network

Kalpesh Prajapati, Vishal M. Chudasama, Heena Patel, Kishor P. Upla, Raghavendra Ramachandra, Kiran B. Raja, Christoph BuschWorkshops @ CVPR 2020 · DOI

Abstract

Traditional Single Image Super-Resolution (SISR) models often fail on real-world images due to their reliance on synthetic, bicubically downsampled training data. We present USISResNet, an unsupervised framework that utilizes a Generative Adversarial Network (GAN) and a novel Quality Assessment (QA) network to super-resolve images with unknown degradations. By incorporating a Mean Opinion Score (MOS) based loss function, USISResNet effectively suppresses real-world noise while restoring high-frequency details, achieving superior perceptual quality on unpaired datasets.

Introduction

The primary challenge in real-world SISR is the discrepancy between synthetic training data and actual sensor output. Most state-of-the-art models are trained on pairs of high-resolution (HR) images and their bicubically downsampled low-resolution (LR) counterparts. However, real-world images suffer from complex degradations, including sensor noise, motion blur, and compression artifacts. Creating perfectly aligned LR-HR pairs for these conditions is impractical. USISResNet addresses this "Blind SR" problem through an unsupervised approach that learns from unpaired data.

Real-world images exhibit complex noise and artifacts that bicubic downsampling fails to model.

Proposed Method: USISResNet

The USISResNet architecture integrates three primary components to balance detail restoration and noise suppression:

  1. Generator (G): Employs 24 Residual-in-Residual (RIR) blocks and Channel Attention (CA) mechanisms. RIR blocks facilitate the learning of complex features, while CA allows the network to prioritize informative feature channels. Global Residual Learning is used to stabilize training by focusing on the difference between the upsampled input and the target HR image.
  2. Discriminator (D): A standard GAN component that encourages the generator to produce realistic textures by attempting to distinguish between generated and real HR images.
  3. Quality Assessment (QA) Network: A CNN pre-trained on the KADID-10K dataset to predict Mean Opinion Scores (MOS). This network acts as a perceptual regularizer.

Loss Function

The model is optimized using a weighted combination of Content Loss, Total Variation (TV) Loss, Adversarial Loss, and a unique QA (MOS) Loss. The MOS loss specifically guides the generator to maximize human-perceived quality, effectively distinguishing between desirable texture and undesirable noise.

Experimental Results

The model was evaluated on the NTIRE-2020 Real-World SR Challenge datasets (Flickr2k for LR and DIV2K for HR).

Perceptual Performance

USISResNet demonstrates significant improvements in perceptual metrics. It achieved an LPIPS score of 0.375, outperforming established models like ESRGAN (0.755) and RCAN (0.576). While GAN-based methods typically score lower on pixel-wise metrics like PSNR and SSIM, the visual fidelity and edge sharpness are markedly superior.

Comparison of USISResNet against baseline methods on real-world noisy data.

Qualitative Analysis

Visual results indicate that USISResNet successfully removes sensor noise and sharpens structural elements, such as building facades and text, which remain blurry or distorted in supervised models. Ablation studies confirmed that the MOS-based loss and TV loss are essential for maintaining structural integrity and suppressing "checkerboard" artifacts.

Conclusion

USISResNet provides a robust solution for super-resolving images with unknown, real-world degradations. By shifting the focus from pixel-perfect reconstruction to human-perceived quality via a MOS-based loss, the framework achieves state-of-the-art perceptual results on unpaired data. Future work will focus on reducing computational complexity and further refining the QA network to handle a broader spectrum of real-world artifacts.

Summary updated

BibTeX
@inproceedings{prajapati2020unsupervised,
  author = {Kalpesh Prajapati and Vishal M. Chudasama and Heena Patel and Kishor P. Upla and Raghavendra Ramachandra and Kiran B. Raja and Christoph Busch},
  title = {{Unsupervised Single Image Super-Resolution Network (USISResNet) for Real-World Data Using Generative Adversarial Network}},
  booktitle = {Workshops @ CVPR},
  year = {2020},
  doi = {10.1109/cvprw50498.2020.00240},
}
Figure