Abstract
The Self-Guided Network (SGN) is a lightweight, high-performance image denoiser designed for real-time use on resource-constrained devices like smartphones. By using a top-down architecture that extracts large-scale context at low resolutions and propagates it to guide high-resolution feature extraction, SGN achieves state-of-the-art accuracy while being up to 20x faster and using 15x less memory than previous complex models.
Introduction
Image denoising—recovering a clean image from a noisy one—is a cornerstone of computer vision. While deep learning has pushed denoising performance to new heights, state-of-the-art (SOTA) models have become increasingly computationally expensive. Models like MemNet or RED produce high-quality results but possess massive memory footprints and high latency, making them unsuitable for real-time applications on mobile hardware found in consumer cameras and smartphones. The Self-Guided Network (SGN) aims to bridge this gap by creating a network that maintains restoration quality without sacrificing speed.
Methodology: Top-Down Self-Guidance
The core innovation of SGN is its Top-Down Self-Guidance architecture. Unlike traditional networks that process images from high to low resolution (bottom-up), SGN reverses this flow to leverage global context efficiently.
Multi-Resolution Shuffling
Instead of standard downsampling methods like pooling or strided convolutions, which can discard fine details, SGN employs a shuffling operation. This operation rearranges pixels into more channels, reducing spatial resolution while preserving all original information. This allows the network to operate at multiple scales without losing the high-frequency data necessary for reconstruction.
Feature Propagation and Guidance
The network begins by processing the lowest-resolution version of the image to gain a "macro-understanding" of the image content. This large-scale context is then passed "down" to higher-resolution sub-networks. These sub-networks use the context as a guide to better extract local features and distinguish noise from actual image texture. By performing the bulk of the computation at lower scales, the full-resolution branches can remain thin and fast.
Experimental Setup
The authors evaluated SGN on several standard benchmarks:
- DIV2K: A high-quality dataset of 800 images for training.
- BSD68: A classic benchmark for evaluating grayscale image denoising.
- SID (See-In-the-Dark): A challenging dataset of raw, short-exposure low-light images used to test performance on real-world sensor data.
Results and Discussion
SGN demonstrates a remarkable "Pareto improvement" over existing methods, optimizing both accuracy and efficiency.
Efficiency and Speed
For a $480 \times 320$ image, SGN takes only 7.5ms on a TITAN Xp GPU. In comparison, the high-performing MemNet takes 156.8ms, making SGN roughly 20 times faster. Furthermore, SGN uses only 0.09 GB of peak GPU memory, a 15x reduction compared to MemNet's 1.37 GB.
Accuracy
Despite its lightweight design, SGN consistently matches or exceeds the Peak Signal-to-Noise Ratio (PSNR) of much heavier models like DnCNN and RED across various noise levels. On the SID dataset, SGN outperformed the popular U-Net architecture by 0.8 dB on the Fuji sensor subset, proving its effectiveness on real-world low-light photography.
Conclusion
SGN proves that understanding global structure at low resolutions makes local denoising significantly more efficient. While the current guidance mechanism uses simple concatenation, future work could explore more sophisticated fusion methods like Spatial Feature Transforms (SFT). Additionally, the self-guidance principle could be extended to other restoration tasks such as super-resolution or deblurring.
Summary updated