Self-Guided Network for Fast Image Denoising

Shuhang Gu, Yawei Li, Luc Van Gool, Radu Timofte

Abstract

The Self-Guided Network (SGN) is a lightweight, high-performance image denoiser designed for real-time use on resource-constrained devices like smartphones. By using a top-down architecture that extracts large-scale context at low resolutions and propagates it to guide high-resolution feature extraction, SGN achieves state-of-the-art accuracy while being up to 20x faster and using 15x less memory than previous complex models.

Introduction

Image denoising—recovering a clean image from a noisy one—is a cornerstone of computer vision. While deep learning has pushed denoising performance to new heights, state-of-the-art (SOTA) models have become increasingly computationally expensive. Models like MemNet or RED produce high-quality results but possess massive memory footprints and high latency, making them unsuitable for real-time applications on mobile hardware found in consumer cameras and smartphones. The Self-Guided Network (SGN) aims to bridge this gap by creating a network that maintains restoration quality without sacrificing speed.

Methodology: Top-Down Self-Guidance

The core innovation of SGN is its Top-Down Self-Guidance architecture. Unlike traditional networks that process images from high to low resolution (bottom-up), SGN reverses this flow to leverage global context efficiently.

The SGN architecture utilizes multi-resolution shuffling and top-down feature propagation to guide denoising.

Multi-Resolution Shuffling

Instead of standard downsampling methods like pooling or strided convolutions, which can discard fine details, SGN employs a shuffling operation. This operation rearranges pixels into more channels, reducing spatial resolution while preserving all original information. This allows the network to operate at multiple scales without losing the high-frequency data necessary for reconstruction.

Feature Propagation and Guidance

The network begins by processing the lowest-resolution version of the image to gain a "macro-understanding" of the image content. This large-scale context is then passed "down" to higher-resolution sub-networks. These sub-networks use the context as a guide to better extract local features and distinguish noise from actual image texture. By performing the bulk of the computation at lower scales, the full-resolution branches can remain thin and fast.

Experimental Setup

The authors evaluated SGN on several standard benchmarks:

  • DIV2K: A high-quality dataset of 800 images for training.
  • BSD68: A classic benchmark for evaluating grayscale image denoising.
  • SID (See-In-the-Dark): A challenging dataset of raw, short-exposure low-light images used to test performance on real-world sensor data.

Results and Discussion

SGN demonstrates a remarkable "Pareto improvement" over existing methods, optimizing both accuracy and efficiency.

Comparison of PSNR vs. Runtime and GPU Memory on the Set68 dataset. SGN achieves competitive accuracy with significantly lower resource requirements.

Efficiency and Speed

For a $480 \times 320$ image, SGN takes only 7.5ms on a TITAN Xp GPU. In comparison, the high-performing MemNet takes 156.8ms, making SGN roughly 20 times faster. Furthermore, SGN uses only 0.09 GB of peak GPU memory, a 15x reduction compared to MemNet's 1.37 GB.

Accuracy

Despite its lightweight design, SGN consistently matches or exceeds the Peak Signal-to-Noise Ratio (PSNR) of much heavier models like DnCNN and RED across various noise levels. On the SID dataset, SGN outperformed the popular U-Net architecture by 0.8 dB on the Fuji sensor subset, proving its effectiveness on real-world low-light photography.

Conclusion

SGN proves that understanding global structure at low resolutions makes local denoising significantly more efficient. While the current guidance mechanism uses simple concatenation, future work could explore more sophisticated fusion methods like Spatial Feature Transforms (SFT). Additionally, the self-guidance principle could be extended to other restoration tasks such as super-resolution or deblurring.

Summary updated

Abstract

During the past years, tremendous advances in image restoration tasks have been achieved using highly complex neural networks. Despite their good restoration performance, the heavy computational burden hinders the deployment of these networks on constrained devices, \eg smart phones and consumer electronic products. To tackle this problem, we propose a self-guided network (SGN), which adopts a top-down self-guidance architecture to better exploit image multi-scale information. SGN directly generates multi-resolution inputs with the shuffling operation. Large-scale contextual information extracted at low resolution is gradually propagated into the higher resolution sub-networks to guide the feature extraction processes at these scales. Such a self-guidance strategy enables SGN to efficiently incorporate multi-scale information and extract good local features to recover noisy images. We validate the effectiveness of SGN through extensive experiments. The experimental results demonstrate that SGN greatly improves the memory and runtime efficiency over state-of-the-art efficient methods, without trading off PSNR accuracy.
BibTeX
@inproceedings{gu2019self,
  author = {Shuhang Gu and Yawei Li and Luc Van Gool and Radu Timofte},
  title = {{Self-Guided Network for Fast Image Denoising}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  doi = {10.1109/iccv.2019.00260},
}
Figure