Cascaded Degradation-Aware Blind Super-Resolution

Ding Zhang, Ni Tang, Dongxiao Zhang, Yanyun Qu2023 · DOI

TL;DR

This paper introduces CDASRN, a novel blind super-resolution (SR) network that tackles the challenge of real-world image degradation by explicitly modeling and estimating spatially varying blur kernels and noise. By decoupling noise estimation from blur kernel estimation and incorporating contrastive learning, CDASRN achieves state-of-the-art performance on both synthetic and real-world datasets, outperforming existing methods in complex degradation scenarios.

What Problem Does This Paper Address?

Single-image super-resolution (SISR) aims to enhance the resolution of low-resolution (LR) images. Traditional SR methods often rely on predefined degradation models for training, which perform poorly when real-world images exhibit different or more complex degradation patterns. Blind SR (BSR) methods attempt to address this by estimating degradation factors (like blur kernels) directly from the LR image. However, existing BSR methods often make simplifying assumptions, such as spatially invariant blur kernels, and neglect the impact of noise on kernel estimation. This leads to suboptimal performance on real-world images, which can have spatially varying blur and significant noise. The paper aims to bridge this gap by proposing a robust BSR method that can handle complex, spatially varying degradations, including noise and blur, without requiring prior knowledge of these factors.

Methods

The proposed method, Cascaded Degradation-Aware Blind Super-Resolution Network (CDASRN), is a cascaded network with three main components:

  • Noise Estimation Subnetwork (CNNn): This module is placed at the beginning of the network to estimate and remove the influence of noise before blur kernel estimation. This is crucial because noise can significantly corrupt blur kernel estimation. The network utilizes a 3-level MWCNN framework with a U-Net structure and discrete wavelet transforms (DWT) to capture image details efficiently.
  • Kernel Estimation Subnetwork (CNNk): This module estimates the spatially varying blur kernel. Unlike previous methods that assume a uniform kernel, CDASRN's CNNk is designed to capture local variations in blur. It employs an encoder-decoder architecture with residual blocks and skip connections. A novel MAConv layer is used to exploit channel interdependence for enhanced feature expressiveness.
  • Degradation-Aware Super-Resolution Subnetwork (CNNsr): This module reconstructs the high-resolution (HR) image. It leverages the estimated blur kernel information by using Principal Component Analysis (PCA) to reduce its dimensionality and then modulates intermediate features within cascaded basic blocks (combining Spatial Feature Transformer - SFT and Residual-in-Residual Dense Blocks - RRDB). Crucially, it incorporates a contrastive learning module to enhance the network's ability to capture local content features by distinguishing between similar and dissimilar image patches.

The overall loss function is a weighted sum of image reconstruction loss (LSR), blur kernel estimation loss (LK), noise estimation loss (LN), and local contrastive learning loss (LLocCon).

Datasets

  • Training: DIV2K and Flickr2K datasets were used for training. A synthetic training set was generated by applying various degradation factors (compression, blur kernel width, noise) to these datasets.
  • Testing: Standard benchmark datasets like Set5, Set14, and B100 were used for testing. These datasets were degraded with different combinations of factors to simulate various scenarios. Real-world images were also used for qualitative evaluation.

Key Insights and Ideas

  • Decoupling Noise from Blur Estimation: The explicit noise estimation subnetwork before blur kernel estimation is a critical insight. It directly addresses how noise can mislead blur kernel estimation, a common pitfall in existing BSR methods.
  • Spatially Varying Blur Kernel Estimation: Recognizing that real-world blur is rarely uniform across an image is key. The CNNk's architecture is designed to capture these local variations, leading to more accurate degradation modeling.
  • Contrastive Learning for Local Features: Incorporating contrastive learning in the SR subnetwork helps the model better distinguish subtle differences in local image content, which is vital for accurate reconstruction, especially in complex degradation scenarios.
  • Cascaded Approach for Robustness: The cascaded design allows each subnetwork to focus on a specific aspect of degradation, leading to a more robust and effective overall system compared to end-to-end approaches that might struggle with multiple, intertwined degradation factors.

Key Results

  • Quantitative Performance: CDASRN achieved state-of-the-art performance on benchmark datasets, outperforming methods like ZSSR, IKC, SFM, DMSR, RCAN, and SRResNet in terms of PSNR and SSIM, especially in challenging anisotropic spatially varying blur scenarios with noise and compression. For instance, in Table 1, CDASRN consistently achieves higher PSNR/SSIM values across various settings.
  • Qualitative Results: Visual comparisons (Figures 6, 7, and 8) demonstrate that CDASRN produces sharper, more detailed, and artifact-free reconstructions compared to other methods, particularly in restoring textures and fine details in complex real-world images.
  • Efficiency: While not explicitly detailed with numbers in the main text, the paper implies that CDASRN is more efficient than iterative methods like IKC, which require significant testing time.

Limitations and Future Work

  • Supervised Nature: The current CDASRN model requires knowledge of the true noise map and blur kernel for training. The authors acknowledge that in real-world scenarios, obtaining these ground truth values is often impossible.
  • Future Work: The paper suggests that future research should focus on unsupervised blind super-resolution methods to overcome the limitations of requiring ground truth degradation information. This would make the approach more practical for diverse real-world applications.

Summary updated

BibTeX
@misc{zhang2023cascaded,
  author = {Ding Zhang and Ni Tang and Dongxiao Zhang and Yanyun Qu},
  title = {{Cascaded Degradation-Aware Blind Super-Resolution}},
  year = {2023},
  doi = {10.3390/s23115338},
}
Figure