Abstract
Deep Convolutional Neural Networks (CNN) have drawn great attention in image super-resolution (SR). Recently, visual attention mechanism, which exploits both of the feature importance and contextual cues, has been introduced to image SR and proves to be effective to improve CNN-based SR performance. In this paper, we make a thorough investigation on the attention mechanisms in a SR model and shed light on how simple and effective improvements on these ideas improve the state-of-the-arts. We further propose a unified approach called "multi-grained attention networks (MGAN)" which fully exploits the advantages of multi-scale and attention mechanisms in SR tasks. In our method, the importance of each neuron is computed according to its surrounding regions in a multi-grained fashion and then is used to adaptively re-scale the feature responses. More importantly, the "channel attention" and "spatial attention" strategies in previous methods can be essentially considered as two special cases of our method. We also introduce multi-scale dense connections to extract the image features at multiple scales and capture the features of different layers through dense skip connections. Ablation studies on benchmark datasets demonstrate the effectiveness of our method. In comparison with other state-of-the-art SR methods, our method shows the superiority in terms of both accuracy and model size.
Introduction
Single Image Super-Resolution (SR) is the challenging task of recovering a high-resolution (HR) image from a single low-resolution (LR) input. While Deep Learning has significantly advanced the field, standard Convolutional Neural Networks (CNNs) often face two primary hurdles: uniform treatment of image regions and scale blindness. Standard convolutions are translation-invariant, meaning they apply the same filters to smooth areas and complex textures alike. Furthermore, many models fail to account for the fact that different features require different receptive fields—large smooth areas benefit from a wider view, while fine details require localized focus.
Proposed Architecture
The Multi-grained Attention Network (MGAN) addresses these issues by introducing a unified attention mechanism within a robust feature extraction framework. The network is divided into two main stages: feature extraction and reconstruction.
Multi-grained Attention Block (MGAB)
The fundamental building block of MGAN is the Multi-Grained Attention Block (MGAB). It incorporates two key structural innovations:
- Multi-scale Dense Connections: Each block uses parallel 3x3 and 5x5 filters to capture features at multiple scales. These are linked via dense skip connections to ensure information preservation and maximize gradient flow during training.
- Multi-grained Attention Mechanism: This core novelty unifies channel and spatial attention. By dividing feature maps into $S \times S$ grids, the model can capture contextual cues at different granularities. When $S=1$, the model performs standard channel attention; as $S$ increases, it approaches spatial attention. By using intermediate values like $S=2$ or $S=4$, the model adaptively re-scales feature responses based on both local and global context.
Experiments and Results
MGAN was trained on the DIV2K dataset and evaluated on five standard benchmarks: Set5, Set14, Bsd100, Urban100, and Manga109. The model demonstrates exceptional performance on datasets with complex structural details, such as Urban100 and Manga109.
Quantitative results indicate that MGAN+ (the self-ensemble version) achieves state-of-the-art PSNR/SSIM across various scales. For example, on Urban100 (x4), MGAN reached 26.47 dB, outperforming the popular MSRN (26.22 dB). Beyond accuracy, MGAN is highly efficient; it achieves these results with approximately 11.7M parameters, roughly one-fourth the size of heavy models like EDSR (~40M+ parameters).
Qualitative comparisons show that MGAN is particularly effective at recovering sharp edges and repetitive patterns in architectural photos, avoiding the distortion or aliasing artifacts common in standard CNN-based approaches.
Conclusion
MGAN advances the state-of-the-art in image super-resolution by unifying channel and spatial attention into a single multi-grained framework. Combined with multi-scale dense connections, the architecture provides a superior performance-to-parameter tradeoff. Future research may focus on automating the selection of optimal grain sizes for different image regions or extending the framework to video super-resolution tasks.
Summary updated