Abstract
Deep convolutional neural networks have made significant progress in image denoising. However, in most cases, denoising methods using a single-stream structure with a single kernel size do not perform so well in integrating complementary contextual information; owing to the lack of this type of information, they may fail to reconstruct fine textures and patterns. To address this problem, we propose a multi-scale gated fusion network (MGFN) for image denoising, which learns direct end-to-end mappings from corrupted images to clean images. Our proposed network consists of several multi-scale mutually-gated (MM) blocks. In each MM block, we incorporate dilated convolution into a merge-and-run (MR) module to exploit multi-scale features in an effective way and further recognize useful features by filtration via a gating mechanism. Moreover, we propose a simple but effective loss function named dropout-loss to train the network. The extensive experiments on benchmark datasets show that our proposed method can well recover textures, yielding favorable performance against other state-of-the-art methods.
Introduction
Image denoising is a fundamental low-level vision task aimed at recovering a clean latent image $x$ from a noisy observation $y = x + n$. While deep learning has revolutionized this field, many popular architectures rely on a single-stream structure with a fixed kernel size. These approaches often struggle to integrate complementary contextual information from different scales, which is essential for reconstructing fine textures and repetitive patterns. If a network only looks at a small local neighborhood, it might mistake a fine texture for noise and blur it away.
Proposed Method: Multi-Scale Gated Fusion Network (MGFN)
The authors introduce the Multi-Scale Gated Fusion Network (MGFN), which is built around a sequence of Multi-scale Mutually-gated (MM) blocks. The architecture features several key innovations designed to preserve image detail while removing noise.
Multi-scale Mutually-gated (MM) Blocks
The core of the MGFN is the MM block, which improves upon standard residual learning through three main mechanisms:
- Modified Merge-and-Run (MR) Mapping: Standard MR modules typically average inputs from parallel branches. MGFN improves this by using a learnable convolution to control how residual information is added, allowing the network to be more selective about what information it preserves.
- Dilated Convolutions for Multi-Scale Learning: To capture wider context without increasing the parameter count, the network uses dilated convolutions in one of its parallel branches. This allows the model to "see" larger patterns (coarse-grained info) in one path while maintaining fine-grained details in the other.
- Mutual Gating Mechanism: This is the "secret sauce" of the MM block. The features from the left branch are used to "gate" (filter) the features of the right branch, and vice versa. This cross-pollination helps the network recognize and emphasize the most useful features for reconstruction.
Dropout-Loss
Inspired by the dropout regularization technique, the authors propose a loss function that randomly masks out pixels during the calculation of the Mean Squared Error (MSE). By only calculating loss on a subset of pixels (using a Bernoulli distribution), the network becomes more robust and avoids overfitting to specific noise patterns.
Experimental Results
The researchers evaluated MGFN across several standard benchmarks for both grayscale and color Gaussian denoising, as well as impulse noise removal, using datasets like BSD68, Set12, and Urban100.
Quantitative and Qualitative Performance
The most significant performance gains were observed on the Urban100 dataset, which is rich in complex structural patterns. Because Urban100 contains many buildings and grids, the multi-scale approach of MGFN excels where single-stream models produce blurry results. For a noise level of $\sigma=50$, MGFN achieved a PSNR of 26.74 dB, outperforming DnCNN (26.28 dB) by a substantial 0.46 dB. Visual comparisons show that MGFN preserves sharp edges and fine repetitive textures that other methods turn into gray smudges.
Efficiency
On an NVIDIA Tesla P100 GPU, MGFN can denoise a $256 \times 256$ image in just 11ms. While slightly slower than ultra-fast models like FFDNet due to its multi-path architecture, it remains highly competitive for real-time applications.
Conclusion
The Multi-Scale Gated Fusion Network addresses the limitations of single-scale denoising models by effectively fusing multi-scale features through a mutual gating mechanism. By combining dilated convolutions with a selective gating process and a novel dropout-loss function, MGFN achieves state-of-the-art results in texture recovery. Future work could involve optimizing the gating mechanism for even greater efficiency or extending the approach to other restoration tasks like super-resolution.
Summary updated