Multi-Scale Feature Learning Convolutional Neural Network for Image Denoising

Shuo Zhang, Chunyu Liu, Yuxin Zhang, Shuai Liu, Xun WangSensors 2023 · DOI

TL;DR

This paper introduces MSFLNet, a novel convolutional neural network for image denoising that effectively removes noise while preserving fine details. It achieves this by employing multi-scale feature learning modules and residual connections to better capture and reconstruct image information across different scales.

What Problem Does This Paper Address?

Images captured in real-world scenarios are often degraded by noise due to hardware limitations and environmental factors. This noise significantly reduces image quality and hinders the performance of subsequent image processing tasks. While existing denoising algorithms can reduce noise, they often struggle to strike a balance between noise removal and the preservation of intricate image details, leading to a loss of important information. This paper aims to address this challenge by developing a denoising method that excels at both noise reduction and detail preservation.

Methods

The proposed method, MSFLNet, is a convolutional neural network built upon three key components:

  • Feature Learning (FL) Modules: These modules are designed to extract image features across multiple scales. They leverage the Res2Net architecture, which allows for the formation of receptive fields of varying sizes to capture fine-grained features. The FL modules also incorporate convolutional layers, batch normalization (BN), and ReLU activation functions to enhance feature extraction capabilities and accelerate network convergence.
  • Reconstruction Generation (RG) Module: This module is responsible for reconstructing a clean image from the features learned by the FL modules. It also utilizes multi-scale feature learning modules and convolutional layers to generate the final denoised output.
  • Residual Connection: This crucial component facilitates the transfer of information from shallower layers of the network to deeper layers. This helps the model combine both local and global image information, improving denoising performance and reducing model complexity.

The overall architecture consists of three FL modules followed by one RG module, with residual connections bridging the gap between them.

Datasets

The paper uses several standard datasets for training and evaluation:

  • Training: Train400, DIV2K, and SIDD. For training, synthetic Gaussian noise was added to images from Train400 and DIV2K at various noise levels (15, 25, and 50), generating a large dataset of noisy-clean image pairs.
  • Testing: BSD68, Set12, TNO, SIDD, and PolyU. These datasets were used to evaluate the performance of MSFLNet against other state-of-the-art denoising algorithms under different noise conditions, including both synthetic and real-world noise.

Key Insights and Ideas

  • Multi-scale feature extraction is key: The paper strongly emphasizes that capturing image information at multiple scales is critical for effective denoising. By using modules like Res2Net, the network can learn features that are sensitive to different levels of detail.
  • Preserving details is as important as removing noise: A significant challenge in denoising is avoiding the loss of fine textures and edges. MSFLNet's architecture, particularly the combination of multi-scale learning and residual connections, is designed to mitigate this.
  • Residual connections bridge the information gap: Transferring shallow features to deeper layers helps the network build a more comprehensive understanding of the image, leading to better reconstruction.
  • Ablation studies confirm component effectiveness: The paper includes ablation experiments that demonstrate the individual contributions of the RG and FL modules, highlighting that their combined use leads to the best performance.

Key Results

MSFLNet demonstrated strong performance across various datasets and noise levels.

  • Quantitative Results: The paper reports that MSFLNet consistently achieved higher Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) values compared to other state-of-the-art denoising algorithms (DnCNN, xUnit, ECNDNet, ADNet, MSANet) on datasets like BSD68, Set12, TNO, SIDD, and PolyU. For instance, on the BSD68 dataset with a noise level of 50, MSFLNet achieved a PSNR of 26.139 and an SSIM of 0.8314, outperforming other methods.
  • Qualitative Results: Visual comparisons presented in the paper (Figures 4, 5, 6, and 7) show that MSFLNet produces denoised images that are notably clearer, with significantly reduced noise and better preservation of fine details and textures compared to its counterparts.

Limitations and Future Work

  • Computational Cost: While the paper claims the model is "relatively reasonable" in terms of parameters and computation (FLOPs), it does not explicitly state it's the most efficient. Future work could focus on further optimizing the model for even greater computational efficiency.
  • Generalizability: The paper focuses on specific types of noise (Gaussian and real-world noise from specific datasets). Further research could explore its effectiveness on a wider range of noise types and more complex image degradation scenarios.
  • Exploration of other multi-scale architectures: While Res2Net is used, exploring other advanced multi-scale feature extraction techniques could potentially lead to further improvements.

Summary updated

Abstract

Affected by the hardware conditions and environment of imaging, images generally have serious noise. The presence of noise diminishes the image quality and compromises its effectiveness in real-world applications. Therefore, in real-world applications, reducing image noise and improving image quality are essential. Although current denoising algorithms can somewhat reduce noise, the process of noise removal may result in the loss of intricate details and adversely impact the overall image quality. Hence, to enhance the effectiveness of image denoising while preserving the intricate details of the image, this article presents a multi-scale feature learning convolutional neural network denoising algorithm (MSFLNet), which consists of three feature learning (FL) modules, a reconstruction generation module (RG), and a residual connection. The three FL modules help the algorithm learn the feature information of the image and improve the efficiency of denoising. The residual connection moves the shallow information that the model has learned to the deep layer, and RG helps the algorithm in image reconstruction and creation. Finally, our research indicates that our denoising method is effective.
BibTeX
@inproceedings{zhang2023multi,
  author = {Shuo Zhang and Chunyu Liu and Yuxin Zhang and Shuai Liu and Xun Wang},
  title = {{Multi-Scale Feature Learning Convolutional Neural Network for Image Denoising}},
  booktitle = {Sensors},
  year = {2023},
  doi = {10.3390/s23187713},
}
Figure