Multi-Scale Selective Feedback Network with Dual Loss for Real Image Denoising

Xiaowan Hu, Yuanhao Cai, Zhihong Liu, Haoqian Wang, Yulun Zhang

Abstract

Real-world image denoising remains a significant challenge due to the complex, sensor-dependent nature of noise and the difficulty of obtaining paired training data. This paper introduces the Multi-Scale Selective Feedback Network (MSFN), a novel architecture that employs an iterative feedback mechanism to refine denoising from coarse to fine scales. By integrating a dual regression strategy, the model effectively learns from both paired and unpaired data, creating a closed-loop supervision system. MSFN achieves state-of-the-art performance on major benchmarks, including SIDD and DND, demonstrating superior visual fidelity and robustness to real-world noise distributions.

Introduction

Image denoising is a classic ill-posed problem where a single noisy input can correspond to multiple potential clean images. Traditional deep learning approaches often struggle with two primary issues: the gap between synthetic Gaussian noise and complex real-world noise, and the "data gap" caused by the scarcity of high-quality paired training sets.

The MSFN addresses these hurdles by mimicking the human visual system's top-down processing. Instead of a simple one-pass prediction, the network uses high-level semantic information to guide low-level feature extraction, ensuring that structural details are preserved while noise is suppressed.

Methodology

The MSFN architecture is built upon three primary innovations designed to enhance feature representation and training efficiency:

Iterative Feedback and Selection

Unlike standard feed-forward networks, MSFN unfolds over multiple time steps ($T$). Deep layers, which capture broad semantic context, pass information back to shallower layers. To prevent these shallow layers from being overwhelmed by redundant data, the authors implemented a Selective Feedback Module (SFM). This module uses Principal Component Analysis (PCA) to compress feedback features, ensuring only the most relevant information guides the next iteration.

Multi-Scale Selective Block (MSB)

To handle features at various granularities, the MSB employs parallel convolutional kernels of different sizes (3x3, 5x5, and 7x7). An internal attention mechanism dynamically weights these scales, allowing the network to prioritize fine details or global context depending on the local image characteristics.

The MSFN architecture featuring iterative feedback unfolding and the dual learning loop.

Dual Regression and Dual Loss

The "dual learning" framework is the core strategy for handling unpaired data. The network simultaneously learns a Primal Task (noisy to clean) and a Dual Task (clean to noisy). This creates a cycle-consistency constraint: a predicted clean image, when re-degraded, should match the original noisy input. This dual loss acts as a powerful regularizer, forcing the model to understand the underlying noise distribution of specific camera sensors.

Experiments and Results

The model was evaluated on both synthetic datasets (DIV2K, Flickr2K) and rigorous real-world benchmarks (SIDD, DND).

Quantitative Performance

MSFN established new state-of-the-art results, notably achieving a PSNR of 40.12 dB on the SIDD benchmark. It consistently outperformed established models like MIRNet and VDN. Furthermore, the unpaired variant (MSFN-U) demonstrated that the dual loss allows the model to adapt to new noise environments without requiring ground-truth clean labels.

Qualitative Analysis

Visually, MSFN excels at recovering sharp edges and fine textures that are often lost to over-smoothing in other models. By leveraging the iterative feedback loop, the network progressively refines the image, successfully reconstructing complex patterns and structural details in challenging real-world photographs.

Example of real-world image denoising, demonstrating the preservation of fine textures and structural clarity.

Discussion and Conclusion

The MSFN demonstrates that bio-inspired feedback mechanisms and dual-task learning are highly effective for real-world image restoration. While the iterative nature of the network increases computational complexity, the performance gains—especially in the absence of paired data—are substantial. Future work may involve adapting this framework to other restoration tasks, such as deblurring or super-resolution, where unpaired training data is common.

Summary updated

Abstract

The feedback mechanism in the human visual system extracts high-level semantics from noisy scenes. It then guides low-level noise removal, which has not been fully explored in image denoising networks based on deep learning. The commonly used fully-supervised network optimizes parameters through paired training data. However, unpaired images without noise-free labels are ubiquitous in the real world. Therefore, we proposed a multi-scale selective feedback network (MSFN) with the dual loss. We allow shallow layers to access valuable contextual information from the following deep layers selectively between two adjacent time steps. Iterative refinement mechanism can remove complex noise from coarse to fine. The dual regression is designed to reconstruct noisy images to establish closed-loop supervision that is training-friendly for unpaired data. We use the dual loss to optimize the primary clean-to-noisy task and the dual noisy-to-clean task simultaneously. Extensive experiments prove that our method achieves state-of-the-art results and shows better adaptability on real-world images than the existing methods.
BibTeX
@inproceedings{hu2021multi,
  author = {Xiaowan Hu and Yuanhao Cai and Zhihong Liu and Haoqian Wang and Yulun Zhang},
  title = {{Multi-Scale Selective Feedback Network with Dual Loss for Real Image Denoising}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.24963/ijcai.2021/101},
}
Figure