Cascading and Enhanced Residual Networks for Accurate Single-Image Super-Resolution

Rushi Lan, Long Sun, Zhenbing Liu, Huimin Lu, Zhixun Su, Cheng Pang, Xiaonan Luo

Abstract

Deep convolutional neural networks (CNNs) have contributed to the significant progress of the single-image super-resolution (SISR) field. However, the majority of existing CNN-based models maintain high performance with massive parameters and exceedingly deeper structures. Moreover, several algorithms essentially have underused the low-level features, thus causing relatively low performance. In this article, we address these problems by exploring two strategies based on novel local wider residual blocks (LWRBs) to effectively extract the image features for SISR. We propose a cascading residual network (CRN) that contains several locally sharing groups (LSGs), in which the cascading mechanism not only promotes the propagation of features and the gradient but also eases the model training. Besides, we present another enhanced residual network (ERN) for image resolution enhancement. ERN employs a dual global pathway structure that incorporates nonlocal operations to catch long-distance spatial features from the the original low-resolution (LR) input. To obtain the feature representation of the input at different scales, we further introduce a multiscale block (MSB) to directly detect low-level features from the LR image. The experimental results on four benchmark datasets have demonstrated that our models outperform most of the advanced methods while still retaining a reasonable number of parameters.

Introduction

Single-Image Super-Resolution (SISR) is a fundamental task in computer vision that aims to reconstruct a high-resolution (HR) image from a single low-resolution (LR) input. While deep learning has revolutionized this field, current state-of-the-art (SOTA) models face three major bottlenecks: parameter bloat due to extremely deep architectures, underutilization of low-level features from the original LR image, and inefficient information propagation as networks grow deeper. This paper addresses these issues by introducing two efficient architectures: the Cascading Residual Network (CRN) and the Enhanced Residual Network (ERN).

Methodology

The core of the proposed approach is the Local Wider Residual Block (LWRB). Unlike standard residual blocks, the LWRB removes Batch Normalization (BN), which can introduce artifacts in super-resolution tasks. Furthermore, it expands the number of channels before the activation function (ReLU), creating a "wider" representation that preserves more information during non-linear processing.

Comparison between (a) a standard residual block with Batch Normalization and (b) the proposed wider residual block without BN.

Building on this block, the authors propose two network strategies:

  1. Cascading Residual Network (CRN): This architecture utilizes "Locally Sharing Groups" (LSGs) connected via a cascading mechanism. Features from earlier groups are fed directly into later groups, ensuring that hierarchical information is reused effectively.
  2. Enhanced Residual Network (ERN): The ERN features a dual global pathway. One path uses a Multiscale Block (MSB) to capture features at various scales using parallel kernel sizes (1x1, 3x3, 5x5). The other path incorporates nonlocal operations to capture long-distance spatial dependencies, which is particularly effective for repetitive textures.
Overview of the proposed network architecture featuring local wider residual blocks (Ni) and multiscale blocks (MSB).

Experiments and Results

The models were trained on the DIV2K dataset and evaluated on four standard benchmarks: Set5, Set14, B100, and Urban100.

Key Findings

  • Efficiency: The proposed models achieve performance nearly identical to the heavyweight EDSR model but with approximately 1/4 the number of parameters. For instance, at a 4x scale, ERN uses ~9.5M parameters compared to EDSR’s ~43M.
  • Quantitative Performance: On the B100 dataset (4x scale), CRN and ERN achieved PSNR/SSIM values of 27.66/0.7395 and 27.70/0.7398 respectively, outperforming models like VDSR and LapSRN.
  • Qualitative Quality: Visual results demonstrate that CRN and ERN are superior at recovering sharp edges and fine structural patterns where other models often produce blurred or aliased artifacts.

Conclusion

By focusing on "wider" rather than just "deeper" architectures and explicitly reusing low-level features, the CRN and ERN provide a highly efficient path to high-quality SISR. These models maintain high representational power while significantly reducing the computational burden. Future work will explore adapting these structures to blind super-resolution and video-based tasks.

Summary updated

Abstract

Deep convolutional neural networks (CNNs) have contributed to the significant progress of the single-image super-resolution (SISR) field. However, the majority of existing CNN-based models maintain high performance with massive parameters and exceedingly deeper structures. Moreover, several algorithms essentially have underused the low-level features, thus causing relatively low performance. In this article, we address these problems by exploring two strategies based on novel local wider residual blocks (LWRBs) to effectively extract the image features for SISR. We propose a cascading residual network (CRN) that contains several locally sharing groups (LSGs), in which the cascading mechanism not only promotes the propagation of features and the gradient but also eases the model training. Besides, we present another enhanced residual network (ERN) for image resolution enhancement. ERN employs a dual global pathway structure that incorporates nonlocal operations to catch long-distance spatial features from the the original low-resolution (LR) input. To obtain the feature representation of the input at different scales, we further introduce a multiscale block (MSB) to directly detect low-level features from the LR image. The experimental results on four benchmark datasets have demonstrated that our models outperform most of the advanced methods while still retaining a reasonable number of parameters.
BibTeX
@article{lan2020cascading,
  author = {Rushi Lan and Long Sun and Zhenbing Liu and Huimin Lu and Zhixun Su and Cheng Pang and Xiaonan Luo},
  title = {{Cascading and Enhanced Residual Networks for Accurate Single-Image Super-Resolution}},
  journal = {IEEE Transactions on Cybernetics},
  year = {2020},
  doi = {10.1109/tcyb.2019.2952710},
}
Figure