Context Reasoning Attention Network for Image Super-Resolution

Yulun Zhang, Donglai Wei, Can Qin, Huan Wang, Hanspeter Pfister, Yun Fu

Introduction

Single Image Super-Resolution (SR) aims to reconstruct a high-resolution (HR) image from a single low-resolution (LR) input. While deep learning has significantly advanced this field, standard Convolutional Neural Networks (CNNs) often struggle with complex, repeating textures due to their reliance on local, static filters. The Context Reasoning Attention Network (CRAN) addresses this by introducing a dynamic mechanism that modulates convolution kernels based on global image context.

The Problem: Local Blind Spots in CNNs

Standard CNNs use $3 \times 3$ filters that only observe small local patches. While feature-based attention mechanisms exist, they typically only rescale feature maps without altering the underlying convolution weights. This limitation makes it difficult for models to capture global structural patterns—such as the consistent orientation of architectural elements—often resulting in blurry or geometrically distorted reconstructions in challenging datasets like Urban100.

Methodology: Context Reasoning Attention

CRAN shifts the paradigm from feature modulation to kernel modulation. The architecture consists of several key stages:

  1. Global Context Extraction: The network compresses input feature maps into global context descriptors.
  2. Context Descriptor Relationship Reasoning (CDRR): Using a graph-based approach, the model calculates pairwise affinities between descriptors to understand semantic relationships across the image.
  3. Dynamic Kernel Modulation: A dynamic attention mask ($F_A$) is generated from the reasoned context. This mask modifies the convolution weights ($W^* = W \odot F_A$) in real-time, allowing the "neurons" to adapt their behavior based on the specific image content.
  4. Efficient Decomposition: To maintain computational efficiency, the attention mask is decomposed into channel and spatial interactions, minimizing the overhead typically associated with non-local operations.

Experiments and Results

The model was evaluated on standard benchmarks including Set5, Set14, B100, Urban100, and Manga109 using Bicubic (BI), Blur-downscale (BD), and Downscale-noise (DN) degradation models.

Visual Fidelity and Performance

CRAN consistently achieves state-of-the-art results, particularly excelling in recovering "structural textures." In architectural scenes, where traditional methods produce wavy lines or artifacts, CRAN maintains sharp, straight edges faithful to the ground truth.

Figure 5: A typical architectural scene from the Urban100 dataset, highlighting a region with complex repeating textures.
Figure 2: The high-resolution reconstruction achieved by CRAN, demonstrating sharp, geometrically accurate grid patterns.

Compared to other "Non-Local" attention methods like CSNLN, CRAN offers a superior balance of performance and efficiency, requiring significantly less GPU memory and offering faster inference times while maintaining higher Peak Signal-to-Noise Ratio (PSNR).

Conclusion

By integrating global semantic reasoning directly into the convolution process, CRAN overcomes the inherent limitations of static local filters. This approach not only improves numerical performance but also significantly enhances the visual realism of reconstructed textures. Future work may extend this context-reasoning kernel modulation to other restoration tasks such as denoising and deblurring.

Summary updated

Abstract

Deep convolutional neural networks (CNNs) are achieving great successes for image super-resolution (SR), where global context is crucial for accurate restoration. However, the basic convolutional layer in CNNs is designed to extract local patterns, lacking the ability to model global context. With global context information, lots of efforts have been devoted to augmenting SR networks, especially by global feature interaction methods. These works incorporate the global context into local feature representation. However, recent advances in neuroscience show that it is necessary for the neurons to dynamically modulate their functions according to context, which is neglected in most CNN based SR methods. Motivated by those observations and analyses, we propose context reasoning attention network (CRAN) to modulate the convolution kernel according to the global context adaptively. Specifically, we extract global context descriptors, which are further enhanced with semantic reasoning. Channel and spatial interactions are then introduced to generate context reasoning attention mask, which is applied to modify the convolution kernel adaptively. Such a modulated convolution layer is utilized as basic component to build the blocks and networks. Extensive experiments on benchmark datasets with multiple degradation models show that CRAN obtains superior results and favorable trade-off between performance and model complexity.
BibTeX
@inproceedings{zhang2021context,
  author = {Yulun Zhang and Donglai Wei and Can Qin and Huan Wang and Hanspeter Pfister and Yun Fu},
  title = {{Context Reasoning Attention Network for Image Super-Resolution}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2021},
  doi = {10.1109/iccv48922.2021.00424},
}
Figure