Consensus Neural Network for Medical Imaging Denoising with Only Noisy Training Samples

Dufan Wu, Kuang Gong, Kyung Sang Kim, Xiang Li, Quanzheng Li

Abstract

Deep neural networks have proven highly efficient for medical image denoising, but standard training paradigms typically require pairs of noisy and "clean" ground-truth images. In many practical medical applications—such as dynamic imaging, spectral CT, or arterial spin labeling MRI—obtaining noise-free ground truth is physically impossible. This paper introduces a "Consensus Neural Network" that learns to denoise using only noisy training samples. By splitting raw acquisition data into two independent noisy subsets and employing a novel consensus loss function, the proposed method achieves performance nearly identical to supervised learning. Validation on Low-dose CT and NYU MRI datasets demonstrates superior performance over existing unsupervised methods like Noise2Noise.

Introduction: The Challenge of Ground Truth in Medical Imaging

Supervised deep learning has revolutionized image denoising, yet its reliance on "clean" reference images remains a significant bottleneck in medical physics. In modalities like low-dose CT or dynamic PET, the signal is inherently stochastic; acquiring a noise-free version would require prohibitive radiation doses or impossible acquisition times.

While the Noise2Noise (N2N) framework demonstrated that models could be trained by mapping one noisy realization to another, it is often impractical for clinical use. N2N typically requires two separate, independent scans of the same anatomy, doubling scan time and patient discomfort. Furthermore, N2N is often inefficient during inference, as it typically utilizes only one noisy realization, leaving valuable signal information unused.

Methodology

The authors propose a framework that overcomes these limitations through two primary innovations:

1. Data Splitting in the Acquisition Domain

Instead of requiring multiple scans, the method splits the raw data from a single scan into two subsets. For CT, this involves partitioning projections (e.g., odd vs. even angles). For MRI, k-space lines are split. These subsets are then reconstructed into two distinct noisy images ($x + n_1$ and $x + n_2$). Because the splitting occurs in the raw domain before reconstruction, the noise in the resulting images remains largely uncorrelated.

2. Consensus Loss and Inference

The core of the approach is a "Consensus Loss" function. Mathematically, the authors provide a formal proof that if noise is zero-mean and uncorrelated between subsets, minimizing the L2 distance between the network's output and the second noisy realization is statistically equivalent to training against a clean ground truth.

During inference, the model processes both noisy subsets and averages the outputs. This "consensus" approach ensures that the final image utilizes all available raw data, leading to better texture preservation and noise reduction than simple mapping.

Experimental Validation

The method was evaluated on two major public datasets:

  • Low-dose CT (LDCT) Grand Challenge: Abdomen CT scans where quarter-dose data was synthesized by adding realistic noise to raw projections.
  • NYU MRI Dataset: Sagittal knee MR scans with simulated $4\times$ undersampling, split into two $8\times$ undersampled subsets for training.

In both experiments, full-dose or fully sampled images were reserved strictly for evaluation purposes.

Key Results and Discussion

The Consensus Neural Network significantly outperformed traditional unsupervised methods and standard Noise2Noise in both quantitative and qualitative metrics.

CT Denoising Performance

In the CT experiments, the proposed method achieved a Structural Similarity Index (SSIM) of 87.1, remarkably close to the 87.7 achieved by supervised "Noise2Clean" training. Standard Noise2Noise lagged behind at 84.2 and exhibited a tendency to oversmooth critical diagnostic features.

Comparison of denoising methods on the Low-dose CT dataset. The proposed consensus method preserves fine details and lesions (blue arrow) better than Noise2noise and TV.

As shown in Figure 1, the consensus method maintains the visibility of small lesions (indicated by the blue arrow) that are blurred by other unsupervised techniques.

MRI Reconstruction

For MRI, the method achieved a Root Mean Square Error (RMSE) of 23.4, compared to 27.7 for Noise2Noise. The consensus framework proved more robust against the aliasing artifacts common in undersampled MRI.

MRI reconstruction results on the NYU knee dataset. The proposed method effectively removes noise and aliasing artifacts while maintaining structural integrity.

Conclusion and Future Directions

The Consensus Neural Network provides a mathematically grounded path toward high-performance denoising without the need for clean ground truth. Its success hinges on the independence of noise achieved by splitting data in the raw acquisition domain.

While highly effective, the method currently requires access to raw projection or k-space data, which is not always available in retrospective clinical studies. Future work may involve integrating this consensus approach into "unrolled" iterative reconstruction networks or developing more sophisticated data consistency terms to further bridge the gap between unsupervised and supervised performance.

Summary updated

Abstract

Deep neural networks have been proved efficient for medical image denoising. Current training methods require both noisy and clean images. However, clean images cannot be acquired for many practical medical applications due to naturally noisy signal, such as dynamic imaging, spectral computed tomography, arterial spin labeling magnetic resonance imaging, etc. In this paper we proposed a training method which learned denoising neural networks from noisy training samples only. Training data in the acquisition domain was split to two subsets and the network was trained to map one noisy set to the other. A consensus loss function was further proposed to efficiently combine the outputs from both subsets. A mathematical proof was provided that the proposed training scheme was equivalent to training with noisy and clean samples when the noise in the two subsets was uncorrelated and zero-mean. The method was validated on Low-dose CT Challenge dataset and NYU MRI dataset and achieved improved performance compared to existing unsupervised methods.
BibTeX
@inproceedings{wu2019consensus,
  author = {Dufan Wu and Kuang Gong and Kyung Sang Kim and Xiang Li and Quanzheng Li},
  title = {{Consensus Neural Network for Medical Imaging Denoising with Only Noisy Training Samples}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2019},
}
Figure