Lightweight attention feature selection recursive network for super-resolution

Wenjie Xu, Huihui Song, Xiao–Tong Yuan, Qingshan LiuJournal of Image and Graphics 2021 · DOI

Abstract

Deep convolutional neural networks have demonstrated exceptional performance in single-image super-resolution (SISR), but the trend toward increasingly complex architectures poses significant challenges for deployment on resource-constrained edge devices. This paper introduces AFSNet, an extremely lightweight and efficient super-resolution network. By employing a recursive feature selection unit and a parameter-sharing mechanism, AFSNet significantly reduces the number of parameters and floating-point operations (FLOPs) while maintaining high reconstruction quality. The network utilizes a recursive Feature Selection Module (FSM) with Efficient Channel Attention (ECA) to extract deep features and a shared upsampling module for final reconstruction. Experimental results on standard benchmarks (Set5, Set14, B100, Urban100, and Manga109) demonstrate that AFSNet achieves superior performance in terms of PSNR and SSIM compared to existing lightweight models, making it highly suitable for real-time applications on mobile hardware.

Introduction

The field of single-image super-resolution (SISR) has been revolutionized by deep learning, with models becoming progressively deeper and wider to capture intricate image details. However, this architectural complexity comes at a high computational cost, limiting the practical application of these models on mobile devices like smartphones. There is a critical need for "lightweight" solutions that balance reconstruction fidelity with efficiency. AFSNet addresses this gap by leveraging recursion and intelligent feature selection to maximize performance per parameter.

Methodology

The AFSNet architecture is structured into three primary stages: shallow feature extraction, deep feature extraction via recursion, and upsampling reconstruction.

1. Recursive Feature Selection Module (FSM)

To minimize the model's footprint, AFSNet adopts a recursive strategy. Instead of a linear stack of unique layers, the network passes features through a single Feature Selection Module (FSM) multiple times (typically six iterations). This approach allows the network to simulate the depth of a much larger model without increasing the total parameter count.

2. Channel Splitting and Feature Enhancement

Within the FSM, a "split-and-merge" strategy is employed to preserve hierarchical information. As features pass through successive convolutional layers, a portion of the channels is set aside (split) and later fused at the end of the module. This mechanism ensures that low-level structural details are not lost during the deep recursive processing, acting as a form of dense residual learning.

3. Efficient Channel Attention (ECA)

To further refine feature maps, an Efficient Channel Attention (ECA) block is integrated. Unlike traditional channel attention mechanisms that use dimensionality reduction—which can inadvertently discard useful information—ECA utilizes a fast 1D convolution to model local cross-channel interactions. This provides a significant boost in representational power with negligible computational overhead.

4. Shared Upsampling Reconstruction

The final reconstruction stage uses a shared upsampling block. By processing both shallow and deep features simultaneously through the same upsampling parameters, the network further optimizes its efficiency while ensuring that both global structure and fine details contribute to the high-resolution output.

Experiments and Results

Datasets

AFSNet was trained on the DF2K dataset (a combination of DIV2K and Flickr2K). Evaluation was conducted on five standard benchmarks: Set5, Set14, B100, Urban100, and Manga109, covering a wide variety of textures and structures.

Performance Comparison

Quantitative analysis shows that AFSNet consistently outperforms other lightweight architectures. For a 4x upscaling task, AFSNet requires only 255K parameters and 24.5G FLOPs. In contrast, models like SRFBN-S require nearly double the parameters (483K) and significantly more FLOPs (852.9G).

On the Set14 (4x) benchmark, AFSNet achieved a PSNR of 28.85 dB, outperforming CARN-M (28.42 dB) and IDN (28.25 dB). Qualitatively, AFSNet excels in reconstructing sharp edges and repetitive patterns, such as those found in urban architecture and fine-lined manga illustrations, where other lightweight models often struggle with blurring.

Conclusion

AFSNet demonstrates that high-quality super-resolution does not necessitate massive, resource-heavy architectures. Through the strategic use of recursion, channel splitting, and efficient attention mechanisms, the model achieves state-of-the-art performance for its size class. This makes AFSNet an ideal candidate for deployment in real-world scenarios requiring efficient, high-fidelity image reconstruction on edge devices.

Summary updated

Abstract

目的 深度卷积网络在图像超分辨率重建领域具有优异性能,越来越多的方法趋向于更深、更宽的网络设计。然而,复杂的网络结构对计算资源的要求也越来越高。随着智能边缘设备(如智能手机)的流行,高效能的超分重建算法有着巨大的实际应用场景。因此,本文提出一种极轻量的高效超分网络,通过循环特征选择单元和参数共享机制,不仅大幅降低了参数量和浮点运算次数(floating point operations,FLOPs),而且具有优异的重建性能。 方法本文网络由浅层特征提取、深层特征提取和上采样重建3部分构成。浅层特征提取模块包含一个卷积层,产生的特征循环经过一个带有高效通道注意力模块的特征选择单元进行非线性映射提取出深层特征。该特征选择单元含有多个卷积层的特征增强模块,通过保留每个卷积层的部分特征并在模块末端融合增强层次信息。通过高效通道注意力模块重新调整各通道的特征。借助循环机制(循环6次)可以有效提升性能且大幅减少参数量。上采样重建通过参数共享的上采样模块同时将浅层与深层特征进放大、融合得到高分辨率图像。 结果与先进的轻量级网络进行对比,本文网络极大减少了参数量和FLOPs,在Set5、Set14、B100、Urban100和Manga109等基准测试数据集上进行定量评估,在图像质量指标峰值信噪比(peak signal to noise ratio,PSNR)和结构相似性(structural similarity,SSIM)上也获得了更好的结果。 结论本文通过循环的特征选择单元有效挖掘出图像的高频信息,并通过参数共享机制极大减少了参数量,实现了轻量的高质量超分重建。
BibTeX
@article{xu2021lightweight,
  author = {Wenjie Xu and Huihui Song and Xiao–Tong Yuan and Qingshan Liu},
  title = {{Lightweight attention feature selection recursive network for super-resolution}},
  journal = {Journal of Image and Graphics},
  year = {2021},
  doi = {10.11834/jig.200555},
}
Figure