Lightweight Non-Local Network for Image Super-Resolution

Risheng Wang, Tao Lei, Wenzheng Zhou, Qi Wang, Hongying Meng, Asoke K. NandiICASSP 2021 · DOI

Abstract

Image Super-Resolution (SR) has traditionally relied on increasingly deep and wide convolutional neural networks, leading to high computational costs and a neglect of spatial relationships. This paper introduces the Lightweight Non-Local Network (LNLN), which utilizes depthwise separable convolutions and non-local modules to capture global spatial context efficiently. LNLN achieves state-of-the-art results while reducing model size from over 55MB to just 8.72MB.

Introduction

Reconstructing high-resolution (HR) images from low-resolution (LR) inputs is an ill-posed problem. While deep learning has significantly improved SR performance, modern architectures like RCAN or EDSR are often too large for edge devices. Furthermore, many models rely on channel attention, which can overlook critical spatial dependencies, resulting in blurred textures. LNLN addresses these challenges by balancing representational power with extreme parameter efficiency.

Architecture and Methodology

The LNLN architecture is designed around the Lightweight Residual Group (LRG), which replaces standard convolutions with more efficient alternatives.

The overall architecture of the Lightweight Non-Local Network (LNLN), featuring depthwise separable convolutions and non-local modules.

Key Components:

  • Non-Local Modules: Unlike standard convolutions that operate on local neighborhoods, non-local modules calculate dependencies between every pixel pair in a feature map. This allows the network to capture long-range spatial information essential for reconstructing repetitive patterns.
  • Depthwise Separable Convolutions (DSC): By splitting standard convolutions into depthwise and pointwise stages, LNLN drastically reduces computational complexity.
  • Regional Grids: To maintain efficiency, non-local operations are performed within regional grids, providing a large receptive field without the quadratic cost of full-image attention.

Experimental Results

LNLN was evaluated on standard benchmarks including Set5, Set14, BSD100, Urban100, and Manga109.

Visual comparison of LNLN against other SR methods, demonstrating sharper edges and fewer artifacts.

Performance and Efficiency

The model demonstrates superior performance on the Urban100 dataset, achieving a PSNR of 26.96 dB at scale factor 4, surpassing the much larger RCAN (26.82 dB). Most notably, LNLN uses only 2.04 million parameters—roughly 14% of the parameter count of previous state-of-the-art modules. This efficiency results in a storage requirement of only 8.72 MB, compared to 55.4 MB for DRCA.

Conclusion

The Lightweight Non-Local Network proves that high-quality super-resolution does not require massive model sizes. By prioritizing spatial context through non-local modules and optimizing convolution types, LNLN offers a viable solution for high-performance SR on resource-constrained devices. Future work will explore semi-supervised learning to further streamline the training process.

Summary updated

BibTeX
@inproceedings{wang2021lightweight,
  author = {Risheng Wang and Tao Lei and Wenzheng Zhou and Qi Wang and Hongying Meng and Asoke K. Nandi},
  title = {{Lightweight Non-Local Network for Image Super-Resolution}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2021},
  doi = {10.1109/icassp39728.2021.9414527},
}
Figure