Abstract
Convolutional Neural Network (CNN)-based image super-resolution (SR) has exhibited impressive success on known degraded low-resolution (LR) images. However, this type of approach is hard to hold its performance in practical scenarios when the degradation process is unknown. Despite existing blind SR methods proposed to solve this problem using blur kernel estimation, the perceptual quality and reconstruction accuracy are still unsatisfactory. In this paper, we analyze the degradation of a high-resolution (HR) image from image intrinsic components according to a degradation-based formulation model. We propose a components decomposition and co-optimization network (CDCN) for blind SR. Firstly, CDCN decomposes the input LR image into structure and detail components in feature space. Then, the mutual collaboration block (MCB) is presented to exploit the relationship between both two components. In this way, the detail component can provide informative features to enrich the structural context and the structure component can carry structural context for better detail revealing via a mutual complementary manner. After that, we present a degradation-driven learning strategy to jointly supervise the HR image detail and structure restoration process. Finally, a multi-scale fusion module followed by an upsampling layer is designed to fuse the structure and detail features and perform SR reconstruction. Empowered by such degradation-based components decomposition, collaboration, and mutual optimization, we can bridge the correlation between component learning and degradation modelling for blind SR, thereby producing SR results with more accurate textures. Extensive experiments on both synthetic SR datasets and real-world images show that the proposed method achieves the state-of-the-art performance compared to existing methods.
Introduction
Single Image Super-Resolution (SISR) aims to recover a high-resolution (HR) image from a single low-resolution (LR) observation. While standard deep learning models excel when the degradation (e.g., bicubic downsampling) is known, they often fail in real-world scenarios where the degradation process is complex and unknown—a challenge known as Blind SR.
Traditional approaches to Blind SR typically rely on explicit blur kernel estimation. However, these methods are prone to error; even slight inaccuracies in the estimated kernel can lead to significant artifacts or over-smoothed results. To address this, we introduce the Components Decomposition and Co-optimization Network (CDCN). Instead of focusing solely on kernel estimation, CDCN models the relationship between the intrinsic components of an image—its structure and its details—to reconstruct high-quality textures and sharp edges.
Methodology
The CDCN architecture is designed to bridge the gap between component learning and degradation physics through three primary modules:
1. Component Decompose Module (CDM)
Rather than processing the LR image as a monolithic input, the CDM decomposes the input features into two distinct streams: Structure and Detail. The structure component captures the broad geometric context and shapes, while the detail component focuses on high-frequency textures and fine edges. This separation allows the network to apply specialized processing to different types of visual information.
2. Mutual Collaboration Block (MCB)
Processing structure and detail in isolation is insufficient for high-fidelity reconstruction. The MCB facilitates interaction between these streams using channel-wise attention. In this collaborative framework, structural features provide the spatial context necessary to locate where details should be enhanced, while detail features provide the high-frequency information required to sharpen structural boundaries. This mutual optimization prevents the "plastic" look common in over-smoothed SR results.
3. Degradation-driven Learning Strategy
We formulate the degradation process mathematically, defining an HR image as the sum of its structural components and the information lost during blurring (the details). Based on this model, we implement a joint supervision strategy using specific loss functions ($L_{structure}$ and $L_{detail}$). These losses ensure that the reconstructed components are mathematically consistent with the physics of image degradation, guiding the network to recover lost information more accurately.
4. Multi-scale Fusion and Reconstruction
To finalize the image, a Multi-scale Fusion Module (MSFM) employs varying convolutional kernel sizes (3x3, 5x5, and 7x7). This allows the model to capture and integrate patterns across different scales before the final upsampling layer produces the HR output.
Experimental Results
Datasets and Training
The model was trained on a combined dataset of DIV2K (800 images) and Flickr2K (2650 images). Evaluation was conducted on standard synthetic benchmarks (Set5, Set14, BSD100, Urban100, and Manga109) under various blur settings, as well as the DIV2KRK dataset, which simulates complex, real-world anisotropic blur.
Quantitative and Qualitative Performance
CDCN demonstrates superior performance across both objective metrics and visual quality:
- Accuracy: On the Set5 benchmark (x4 scale), CDCN achieved a PSNR of 32.04 dB, surpassing other leading blind SR methods such as DAN (31.89 dB) and DSSR (31.97 dB).
- Visual Fidelity: Qualitatively, CDCN produces significantly sharper edges and more realistic textures, particularly in challenging areas like text and repetitive patterns, where iterative kernel-based methods often produce smudging.
- Efficiency: CDCN offers a favorable trade-off between speed and quality. It processes a x2 SR image in approximately 0.19 seconds, making it over 15 times faster than iterative methods like IKC (3.01 seconds).
Conclusion
By shifting the focus from explicit kernel estimation to the co-optimization of image components, CDCN provides a robust framework for Blind SR. The integration of a degradation-driven learning strategy ensures that the restoration process remains faithful to the underlying physics of image loss. Future work will explore mining internal image information to handle even more complex, non-linear degradations and extending this component-based approach to video super-resolution.
Summary updated