MAANet: Multi-view Aware Attention Networks for Image Super-Resolution

Jingcai Guo, Shiheng Ma, Song GuoarXiv 2019 · arXiv

Abstract

This paper introduces Multi-view Aware Attention Networks (MAANet), a novel deep learning architecture designed for single-image super-resolution (SR). By integrating specialized local and global attention mechanisms, MAANet explicitly prioritizes scarce high-frequency details, such as sharp edges and fine textures, which are often lost in standard convolutional neural networks. The architecture employs a Local Aware (LA) attention module to amplify high-frequency residuals and a Global Aware (GA) attention module to re-weight feature maps based on learned spatial statistics. Experimental results demonstrate that MAANet achieves state-of-the-art performance in both quantitative metrics and qualitative visual reconstruction, particularly at challenging 4x scaling factors.

Introduction

Image super-resolution (SR) aims to reconstruct a high-resolution (HR) image from a low-resolution (LR) input. While Deep Convolutional Neural Networks (DCNNs) have significantly advanced the field, they often suffer from a fundamental limitation: the equal treatment of all extracted features. LR images are typically dominated by redundant low-frequency information, such as smooth gradients, while critical high-frequency details are sparse. Standard DCNNs optimized with pixel-wise losses (e.g., L1 or L2) tend to prioritize these dominant low-frequency components, resulting in overly smooth or blurry reconstructions. Furthermore, as networks grow deeper to capture complex mappings, they frequently encounter training bottlenecks like vanishing gradients. MAANet is proposed to address these challenges by explicitly focusing on high-frequency detail recovery and enhancing feature reuse through a multi-stage attention-driven architecture.

Proposed Methodology

MAANet utilizes a five-stage pipeline: shallow extraction, deep extraction, upscaling, global attention, and reconstruction. The core innovations reside in its attention modules and block design:

Local Aware (LA) Attention

Operating in the low-resolution feature space, the LA module serves as a high-frequency amplifier. It calculates the "average smoothness" of a feature map region via average pooling and subsequent upsampling. By subtracting this smooth baseline from the original feature map, the module isolates the high-frequency residual. This residual is then scaled by a hyperparameter $\beta$ and added back to the original map to highlight sharp details.

Figure 3: Detailed schematic of the Local Aware (LA) Attention module, illustrating the high-frequency residual extraction process.

Global Aware (GA) Attention

Applied after the upscale unit in the high-resolution space, the GA module re-weights feature channels based on their relative importance. Unlike standard channel attention mechanisms that rely on simple global average pooling, GA attention utilizes a trainable spatial filter to gather complex statistics across feature maps, providing the network with greater expressive power.

LARD Block and Architecture Refinements

To facilitate deep scaling, the authors introduce the Local Attentive Residual-Dense (LARD) block. This structure combines LA attention with dense and residual skip connections to maximize feature reuse and mitigate vanishing gradients. Notably, the architecture omits Batch Normalization (BN) layers. The authors argue that BN can distort original image contrast and color distributions, which is detrimental to the fidelity required for SR tasks.

Experimental Results

Datasets and Training

MAANet was trained on the DIV2K dataset (800 HR images). Evaluation was conducted across five standard benchmarks: Set5, Set14, BSDS100, Urban100 (noted for grid patterns), and Manga109 (noted for sharp lines and text).

Quantitative and Qualitative Performance

MAANet was benchmarked against seven state-of-the-art algorithms, including RDN and DBPN.

  • Quantitative: MAANet consistently outperformed competitors across all datasets for both 2x and 4x upscaling. For instance, on Set5 at 4x scale, MAANet achieved a PSNR of 32.85 dB, significantly higher than the 32.47 dB achieved by RDN.
  • Qualitative: Visually, MAANet excels in reconstructing complex geometric structures. In tests on Urban100 and Manga109, the model successfully recovered distinct parallel lines and sharp text where other models produced blurred artifacts or merged details.

Conclusion and Future Work

The success of MAANet underscores the importance of unequal feature treatment in super-resolution. By replacing standard pooling with trainable filters and explicitly isolating high-frequency residuals, the network achieves superior detail preservation. While the current model relies on a manually tuned hyperparameter ($\beta = 0.07$) for local attention, future iterations could explore making this parameter learnable to further enhance the network's adaptive capabilities across diverse image types.

Summary updated

Abstract

In most recent years, deep convolutional neural networks (DCNNs) based image super-resolution (SR) has gained increasing attention in multimedia and computer vision communities, focusing on restoring the high-resolution (HR) image from a low-resolution (LR) image. However, one nonnegligible flaw of DCNNs based methods is that most of them are not able to restore high-resolution images containing sufficient high-frequency information from low-resolution images with low-frequency information redundancy. Worse still, as the depth of DCNNs increases, the training easily encounters the problem of vanishing gradients, which makes the training more difficult. These problems hinder the effectiveness of DCNNs in image SR task. To solve these problems, we propose the Multi-view Aware Attention Networks (MAANet) for image SR task. Specifically, we propose the local aware (LA) and global aware (GA) attention to deal with LR features in unequal manners, which can highlight the high-frequency components and discriminate each feature from LR images in the local and the global views, respectively. Furthermore, we propose the local attentive residual-dense (LARD) block, which combines the LA attention with multiple residual and dense connections, to fit a deeper yet easy to train architecture. The experimental results show that our proposed approach can achieve remarkable performance compared with other state-of-the-art methods.
BibTeX
@misc{guo2019maanet,
  author = {Jingcai Guo and Shiheng Ma and Song Guo},
  title = {{MAANet: Multi-view Aware Attention Networks for Image Super-Resolution}},
  year = {2019},
  eprint = {1904.06252},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1904.06252},
}
Figure