A Multi-Head Convolutional Neural Network With Multi-path Attention improves Image Denoising

Jiahong Zhang, Meijun Qu, Ye Wang, Lihong Cao

Abstract

Recently, convolutional neural networks (CNNs) and attention mechanisms have been widely used in image denoising and achieved satisfactory performance. However, the previous works mostly use a single head to receive the noisy image, limiting the richness of extracted features. Therefore, a novel CNN with multiple heads (MH) named MHCNN is proposed in this paper, whose heads will receive the input images rotated by different rotation angles. MH makes MHCNN simultaneously utilize features of rotated images to remove noise. To integrate these features effectively, we present a novel multi-path attention mechanism (MPA). Unlike previous attention mechanisms that handle pixel-level, channel-level, or patch-level features, MPA focuses on features at the image level. Experiments show MHCNN surpasses other state-of-the-art CNN models on additive white Gaussian noise (AWGN) denoising and real-world image denoising. Its peak signal-to-noise ratio (PSNR) results are higher than other networks, such as BRDNet, RIDNet, PAN-Net, and CSANN.

Introduction

Image denoising is a fundamental inverse problem in computer vision, aiming to recover a clean image from its noisy observation. While Convolutional Neural Networks (CNNs) have significantly advanced the state-of-the-art, most existing architectures process noisy inputs from a single perspective. This single-head approach often fails to capture the full richness of image features, as textures and edges are inherently orientation-dependent, whereas noise typically remains isotropic.

To address this, we propose the Multi-Head Convolutional Neural Network (MHCNN). By processing the same noisy image at multiple rotation angles (0°, 90°, and 180°) simultaneously, the network can extract more robust features. This multi-perspective approach is complemented by a novel Multi-path Attention (MPA) mechanism designed to fuse these diverse features at the image level.

The MHCNN Architecture

The MHCNN framework consists of three primary stages: multi-head feature extraction, multi-path attention-based fusion, and a noise processing module.

Multi-Head Feature Extraction

The input noisy image is replicated and rotated to create three distinct views: 0°, 90°, and 180°. Each view is processed by a dedicated "Path_Block." These blocks utilize a densely connected structure to maximize feature reuse and gradient flow. By analyzing different orientations, the network can better distinguish structural signals from random noise.

Multi-path Attention (MPA)

The core innovation of MHCNN is the MPA module. While standard attention mechanisms operate on pixels or channels, MPA operates at the image level. It aligns the feature maps from the rotated heads back into the original coordinate space using matrix operations and instance normalization. This ensures that features extracted from different orientations are spatially synchronized before fusion.

Noise Processing and Residual Learning

Following feature fusion, the network employs an Efficient Channel Attention (ECA) layer and a "Tail" block to refine the feature representation. The model follows the principle of residual learning, predicting the noise component rather than the clean image directly. The final denoised output is obtained by subtracting the estimated noise from the original noisy input.

The overall architecture of MHCNN, featuring the Multi-Head (MH) module, Multi-path Attention (MPA), and detailed views of the Path_Block and Tail components.

Experimental Results

The performance of MHCNN was rigorously evaluated on both synthetic (AWGN) and real-world noise datasets.

Quantitative Performance

On standard benchmarks like Set12, Kodak24, and McMaster, MHCNN consistently achieved higher Peak Signal-to-Noise Ratio (PSNR) values compared to established models like DnCNN, FFDNet, and BRDNet. For instance, at a high noise level (σ=50) on the Set12 dataset, MHCNN reached 27.70 dB, significantly outperforming DnCNN's 27.18 dB. In real-world scenarios using the DND benchmark, MHCNN achieved a state-of-the-art PSNR of 39.52 dB.

Qualitative Analysis

Visual comparisons demonstrate that MHCNN excels at preserving fine details and sharp edges that are often lost or blurred by other denoisers. As shown in the butterfly example, MHCNN recovers intricate textures more faithfully than competing methods.

Visual comparison of denoising results on a butterfly image. (a) Original image, (b) Noisy image (σ=50), (c) DnCNN, (d) FFDNet, (e) BRDNet, (f) Proposed MHCNN.

Conclusion

MHCNN introduces a powerful paradigm shift by treating image rotation as a simultaneous architectural feature rather than just a data augmentation technique. The combination of multi-head extraction and image-level multi-path attention allows the model to achieve superior denoising performance across various noise types. While the multi-head approach increases computational complexity, the gains in image fidelity make it a compelling choice for high-quality restoration tasks.

Summary updated

Abstract

Recently, convolutional neural networks (CNNs) and attention mechanisms have been widely used in image denoising and achieved satisfactory performance. However, the previous works mostly use a single head to receive the noisy image, limiting the richness of extracted features. Therefore, a novel CNN with multiple heads (MH) named MHCNN is proposed in this paper, whose heads will receive the input images rotated by different rotation angles. MH makes MHCNN simultaneously utilize features of rotated images to remove noise. To integrate these features effectively, we present a novel multi-path attention mechanism (MPA). Unlike previous attention mechanisms that handle pixel-level, channel-level, or patch-level features, MPA focuses on features at the image level. Experiments show MHCNN surpasses other state-of-the-art CNN models on additive white Gaussian noise (AWGN) denoising and real-world image denoising. Its peak signal-to-noise ratio (PSNR) results are higher than other networks, such as BRDNet, RIDNet, PAN-Net, and CSANN. The code is accessible at https://github.com/JiaHongZ/MHCNN.
BibTeX
@misc{zhang2022multi,
  author = {Jiahong Zhang and Meijun Qu and Ye Wang and Lihong Cao},
  title = {{A Multi-Head Convolutional Neural Network With Multi-path Attention improves Image Denoising}},
  year = {2022},
  eprint = {2204.12736},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2204.12736},
}
Figure