Abstract
Learning from unlabeled and noisy data is one of the grand challenges of machine learning. In this work, we revisit a classical idea: Stein's Unbiased Risk Estimator (SURE). We show that, in the context of image recovery, SURE and its generalizations can be used to train convolutional neural networks (CNNs) for a range of image denoising and recovery problems without any ground truth data. Specifically, our goal is to reconstruct an image $x$ from a noisy linear transformation of the image. We consider scenarios where no additional data is available or where we only have access to other noisy measurements. We demonstrate that SURE can estimate the mean-squared-error (MSE) loss associated with an estimate of $x$, allowing us to train networks for denoising and compressed sensing. Furthermore, we use the SURE framework to explain and improve upon the "Deep Image Prior" phenomenon, where a network fit to a single noisy image can effectively denoise it.
The Ground Truth Bottleneck
Deep learning has revolutionized image recovery tasks like denoising and super-resolution. However, training these models typically requires massive datasets of paired images: a corrupted input and a perfectly clean "ground truth" target. In many real-world domains—such as medical imaging (MRI, CT), microscopy, and astronomy—capturing noise-free ground truth data is either physically impossible or prohibitively expensive. This paper addresses this bottleneck by asking: Can we train state-of-the-art image recovery networks using only noisy, unlabeled measurements?
Methodology: Training with SURE
The core engine of this approach is Stein’s Unbiased Risk Estimator (SURE). SURE is a statistical theorem that provides an unbiased estimate of the MSE of an estimator, even when the true underlying signal is unknown, provided the noise is Gaussian. The SURE formula balances two components:
- Data Fidelity: How well the network's output matches the noisy input.
- Divergence: How sensitive the network's output is to tiny changes in the input.
Because calculating the exact divergence of a deep neural network is computationally intensive, the authors employ Monte-Carlo SURE (MC-SURE). This method approximates the divergence using a single random noise perturbation, making it feasible to use as a loss function for training CNNs.
Preventing Overfitting
A key insight of this work is how the divergence term in SURE acts as a natural regularizer. In the context of "Deep Image Prior" (DIP)—where a network is optimized on a single noisy image—standard loss functions eventually cause the network to memorize the noise, leading to a degradation in image quality (overfitting).
As shown in the figure above, when minimizing only data fidelity, the true error (NMSE) begins to rise after approximately 40 iterations. By contrast, the SURE loss tracks the true NMSE closely and prevents the network from fitting the noise, eliminating the need for arbitrary early stopping.
Key Results
The authors applied SURE-based training to three scenarios: untrained single-image denoising, training on noisy datasets, and compressive sensing.
- Untrained Denoising: A U-Net optimized on a single noisy image using SURE achieved a PSNR of 26.3 dB on the Mandrill image, matching the performance of the classic BM3D algorithm.
- Noisy Dataset Training: A DnCNN trained exclusively on noisy images from the BSD-500 dataset performed nearly as well as a model trained on clean images (e.g., 29.0 dB vs. 29.4 dB on the Barbara image).
- Compressive Sensing: The LDAMP network trained with SURE on undersampled measurements achieved performance on par with BM3D-AMP while running significantly faster at test time.
Conclusion
By leveraging Stein's Unbiased Risk Estimator, this research demonstrates that ground truth data is not a strict requirement for training high-performance image recovery networks. The inclusion of a divergence penalty not only allows for unsupervised training but also provides a theoretical foundation for understanding and improving self-supervised techniques like Deep Image Prior. While the method introduces some computational overhead, it opens new doors for deep learning applications in fields where clean data is a luxury.
Summary updated