Spatial-Adaptive Network for Single Image Denoising

Meng Chang, Qi Li, Huajun Feng, Zhihai Xu

Abstract

Previous works have shown that convolutional neural networks can achieve good performance in image denoising tasks. However, limited by the local rigid convolutional operation, these methods lead to oversmoothing artifacts. A deeper network structure could alleviate these problems, but more computational overhead is needed. In this paper, we propose a novel spatial-adaptive denoising network (SADNet) for efficient single image blind noise removal. To adapt to changes in spatial textures and edges, we design a residual spatial-adaptive block. Deformable convolution is introduced to sample the spatially correlated features for weighting. An encoder-decoder structure with a context block is introduced to capture multiscale information. With noise removal from the coarse to fine, a high-quality noise-free image can be obtained. We apply our method to both synthetic and real noisy image datasets. The experimental results demonstrate that our method can surpass the state-of-the-art denoising methods both quantitatively and visually.

Introduction

Image denoising is a fundamental task in computer vision, aiming to recover a clean image from its noisy version. While deep learning models, particularly Convolutional Neural Networks (CNNs), have significantly advanced the state-of-the-art, they often struggle with preserving fine details and textures. A primary limitation is the "rigidity" of standard convolutional kernels, which operate on fixed-grid sampling patterns. This lack of spatial adaptability often results in oversmoothing, where sharp edges and intricate textures are blurred alongside the noise.

Furthermore, standard CNNs require deep architectures or extensive downsampling to achieve a sufficient receptive field, which increases computational complexity and can lead to information loss. To address these challenges, we present the Spatial-Adaptive Denoising Network (SADNet). SADNet is designed for efficient, blind image denoising by incorporating spatial adaptability directly into the network's architecture. By leveraging modulated deformable convolutions, the network can dynamically adjust its sampling locations based on the local image content, effectively distinguishing between noise and structural details.

Methodology

The SADNet architecture follows an encoder-decoder structure, optimized for multiscale feature extraction and refinement. It processes noise removal from a coarse to fine scale, ensuring that global context informs local detail restoration.

The overall architecture of the proposed Spatial-Adaptive Denoising Network (SADNet).

Residual Spatial-Adaptive Block (RSAB)

The core innovation of SADNet is the Residual Spatial-Adaptive Block (RSAB). Unlike standard residual blocks that use fixed $3 \times 3$ convolutions, the RSAB employs modulated deformable convolutions. This allows the network to learn spatial offsets for each sampling point, enabling the kernel to "deform" and align with local image structures like edges and textures. By sampling from spatially correlated regions, the RSAB can more effectively aggregate relevant information while ignoring noise.

Detailed structure of the Residual Spatial-Adaptive Block (RSAB) and the mechanism of deformable convolution.

Context Block and Offset Transfer

At the bottleneck of the encoder-decoder, a Context Block is integrated to capture long-range dependencies. It utilizes multiple parallel dilated convolutions with varying dilation rates, expanding the receptive field without sacrificing spatial resolution.

To further enhance the accuracy of the deformable sampling, SADNet implements an Offset Transfer mechanism. Offsets learned at coarser scales are upsampled and transferred to finer scales. This coarse-to-fine strategy ensures that the network maintains a consistent understanding of image geometry across different resolutions, which is particularly beneficial when dealing with high levels of noise that might obscure local structures.

Experiments and Results

We evaluated SADNet on a comprehensive set of benchmarks, including synthetic noise (AWGN on BSD68 and Kodak24) and real-world noise (DND, SIDD, Nam, and Poly). These datasets contain actual sensor noise from various cameras, which is significantly more challenging than synthetic noise.

Quantitative Performance

SADNet demonstrates superior performance compared to existing state-of-the-art methods. On the DND benchmark, SADNet achieved a Peak Signal-to-Noise Ratio (PSNR) of 39.59 dB, significantly outperforming RIDNet (39.26 dB) and CBDNet (38.06 dB). Similar gains were observed on the SIDD validation set, where SADNet reached 39.46 dB, compared to 38.71 dB for RIDNet.

Efficiency and Speed

Despite its high performance, SADNet remains computationally efficient. The use of multiscale processing allows complex operations to occur on downsampled feature maps, keeping the total Floating Point Operations (FLOPs) at 50.1G. In terms of inference speed, SADNet processes a $480 \times 320$ color image in approximately 26.7ms, making it roughly 3x faster than RIDNet and nearly 40x faster than RNAN.

Conclusion

SADNet introduces a powerful spatial-adaptive mechanism for image denoising. By replacing rigid convolutions with modulated deformable ones and employing a multiscale encoder-decoder architecture with offset transfer, the network achieves a balance between noise removal and detail preservation. Experimental results on both synthetic and real-world datasets confirm that SADNet provides state-of-the-art performance with high computational efficiency, offering a robust solution for practical image restoration tasks.

Summary updated

Abstract

Previous works have shown that convolutional neural networks can achieve good performance in image denoising tasks. However, limited by the local rigid convolutional operation, these methods lead to oversmoothing artifacts. A deeper network structure could alleviate these problems, but more computational overhead is needed. In this paper, we propose a novel spatial-adaptive denoising network (SADNet) for efficient single image blind noise removal. To adapt to changes in spatial textures and edges, we design a residual spatial-adaptive block. Deformable convolution is introduced to sample the spatially correlated features for weighting. An encoder-decoder structure with a context block is introduced to capture multiscale information. With noise removal from the coarse to fine, a high-quality noisefree image can be obtained. We apply our method to both synthetic and real noisy image datasets. The experimental results demonstrate that our method can surpass the state-of-the-art denoising methods both quantitatively and visually.
BibTeX
@inproceedings{chang2020spatial,
  author = {Meng Chang and Qi Li and Huajun Feng and Zhihai Xu},
  title = {{Spatial-Adaptive Network for Single Image Denoising}},
  booktitle = {European Conference on Computer Vision},
  year = {2020},
  url = {https://openreview.net/forum?id=Jt4bxsT2Dpj},
}
Figure