Spatial-Adaptive Network for Single Image Denoising

Meng Chang, Qi Li, Huajun Feng, Zhihai Xu

Abstract

Previous works have shown that convolutional neural networks can achieve good performance in image denoising tasks. However, limited by the local rigid convolutional operation, these methods lead to oversmoothing artifacts. A deeper network structure could alleviate these problems, but at the cost of additional computational overhead. In this paper, we propose a novel spatial-adaptive denoising network (SADNet) for efficient single image blind noise removal. To adapt to changes in spatial textures and edges, we design a residual spatial-adaptive block. Deformable convolution is introduced to sample the spatially related features for weighting. An encoder-decoder structure with a context block is introduced to capture multiscale information. By conducting noise removal from coarse to fine, a high-quality noise-free image is obtained. We apply our method to both synthetic and real noisy image datasets. The experimental results demonstrate that our method outperforms the state-of-the-art denoising methods both quantitatively and visually.

Introduction

Image denoising is a fundamental computer vision task that faces a persistent trade-off: removing noise often results in the accidental erasure of fine details like textures and sharp edges. Standard convolutional neural networks (CNNs) are often limited by "rigid" convolutions that use fixed-location neighborhoods (e.g., a 3x3 grid). When a pixel is near an edge, these rigid grids pull information from both sides, leading to undesirable blurring and oversmoothing artifacts.

To overcome these limitations, we propose the Spatial-Adaptive Denoising Network (SADNet). SADNet moves away from rigid grids by utilizing modulated deformable convolutions, allowing the network to adapt its sampling patterns to the actual textures and edges of an image. This approach effectively removes heavy noise while preserving the structural integrity of the scene.

Proposed Method: SADNet

The architecture of SADNet is built upon an encoder-decoder structure designed to capture multiscale information and perform noise removal from coarse to fine resolutions. The network incorporates three primary innovations:

  1. Residual Spatial-Adaptive Block (RSAB): Instead of standard convolutions, the RSAB employs modulated deformable convolutions. This allows the network to learn "offsets" for its sampling points. Near edges, the sampling points physically stretch and align along the edge rather than crossing it, preventing blur.
  2. Encoder-Decoder with Offset Transfer: The network estimates sampling offsets at a coarse scale and "transfers" them to finer scales. This coarse-to-fine strategy helps the model locate relevant features more accurately across different resolutions.
  3. Context Block: Located at the bottleneck, this block uses multiple dilated convolutions with varying rates to expand the receptive field without the information loss typically associated with further downsampling.
The overall architecture of SADNet, showcasing the encoder-decoder structure, RSAB modules, and the bottleneck Context Block.

Experimental Results

SADNet was evaluated on both synthetic (DIV2K, BSD68, Kodak24) and real-world datasets (SIDD, DND, RENOIR). Quantitatively, SADNet achieved state-of-the-art performance. On the DND benchmark, it reached a PSNR of 39.59 dB, significantly outperforming established models like RIDNet (39.26 dB). On the SIDD validation set, it achieved 39.46 dB.

Beyond accuracy, SADNet is highly efficient. While it contains 4.3M parameters, its computational load is low (50.1G FLOPs) because complex operations are performed on downsampled feature maps. It processed a 480x320 image in just 26.7 ms, nearly 40 times faster than high-performing non-local models like RNAN.

An ablation study confirms that each architectural component—the RSAB, the offset transfer mechanism, and the Context block—contributes significantly to the final denoising performance.

Ablation study results demonstrating the incremental PSNR gains achieved by incorporating RSAB, offset transfer, and the Context block.

Conclusion

SADNet introduces an effective spatial-adaptive approach to image denoising. By learning to adjust sampling patterns based on local image geometry, the model avoids the oversmoothing common in rigid CNNs. The combination of deformable convolutions, coarse-to-fine offset transfer, and a multiscale context block allows for high-quality restoration that is both computationally efficient and visually superior to existing methods. Future work may explore the application of these spatial-adaptive blocks to other restoration tasks such as deblurring and super-resolution.

Summary updated

Abstract

Previous works have shown that convolutional neural networks can achieve good performance in image denoising tasks. However, limited by the local rigid convolutional operation, these methods lead to oversmoothing artifacts. A deeper network structure could alleviate these problems, but at the cost of additional computational overhead. In this paper, we propose a novel spatial-adaptive denoising network (SADNet) for efficient single image blind noise removal. To adapt to changes in spatial textures and edges, we design a residual spatial-adaptive block. Deformable convolution is introduced to sample the spatially related features for weighting. An encoder-decoder structure with a context block is introduced to capture multiscale information. By conducting noise removal from coarse to fine, a high-quality noise-free image is obtained. We apply our method to both synthetic and real noisy image datasets. The experimental results demonstrate that our method outperforms the state-of-the-art denoising methods both quantitatively and visually.
BibTeX
@inproceedings{chang2020spatial,
  author = {Meng Chang and Qi Li and Huajun Feng and Zhihai Xu},
  title = {{Spatial-Adaptive Network for Single Image Denoising}},
  booktitle = {European Conference on Computer Vision},
  year = {2020},
  url = {https://openreview.net/forum?id=Jt4bxsT2Dpj},
}
Figure