Abstract
Recent advancements in deep learning have enabled image denoising without clean reference data, yet existing methods like Noise2Noise, Noise2Void, and Stein’s Unbiased Risk Estimator (SURE) lack a unified mathematical framework. We present Noise2Score, a novel approach that bridges these disparate techniques using Tweedie’s formula. By estimating the score function (the gradient of the log-likelihood) from noisy images via an amortized residual denoising autoencoder, Noise2Score provides a universal solution for any noise distribution within the exponential family. Our experiments demonstrate that Noise2Score significantly outperforms state-of-the-art self-supervised methods on benchmark datasets, achieving performance levels comparable to supervised learning.
Introduction
In many high-stakes imaging domains, such as medical diagnostics, satellite surveillance, and microscopy, obtaining clean "ground truth" images for training denoising models is often impossible. This has led to the rise of self-supervised denoising, where models learn directly from noisy observations. However, the field has been characterized by a collection of specialized heuristics: Noise2Noise requires multiple noisy realizations of the same scene, while Noise2Void employs "blind-spot" architectures to avoid learning identity mappings.
Noise2Score addresses this fragmentation by identifying a "missing link" in the mathematical structure of self-supervised learning. By leveraging Tweedie’s formula, the authors provide a principled Bayesian framework that treats denoising as a posterior mean estimation problem, unified by the concept of the "score function."
Theoretical Foundation: Tweedie’s Formula
The core innovation of Noise2Score is the application of Tweedie’s formula, a classical result in Bayesian statistics. This formula allows for the calculation of the posterior mean—the optimal estimate of a clean image—using only the noisy observation and the score function of the noisy data distribution.
Crucially, the score function (the gradient of the log-likelihood) captures the underlying structure of the image data. The authors demonstrate that the training objectives of popular methods like Noise2Void are essentially approximations of this score function. By making this relationship explicit, Noise2Score moves self-supervised denoising from a set of architectural tricks to a rigorous statistical inference problem.
The Noise2Score Pipeline
The Noise2Score method decouples the learning of image structures from the specific characteristics of the noise. The process follows a two-step pipeline:
- Universal Score Estimation (Training): An Amortized Residual Denoising Autoencoder (AR-DAE) is trained on noisy images. The network learns to estimate the score function by removing small amounts of synthetic noise added during training. This phase is "universal" because the network learns the prior distribution of the images, which is independent of whether the final target noise is Gaussian, Poisson, or Gamma.
- Noise-Specific Inference: During the inference phase, the pre-trained score function is plugged into the specific version of Tweedie’s formula that corresponds to the noise type at hand. This allows a single trained model to be adapted to various noise models (e.g., Gaussian for electronics, Poisson for photon counting) without retraining.
Experimental Results
The authors evaluated Noise2Score against standard benchmarks (BSD68, Set12, CBSD68, and Kodak) and real-world microscopy data (FMD).
- Quantitative Performance: Noise2Score consistently outperformed existing self-supervised baselines, including Noise2Void and SURE. In many benchmarks, it achieved a PSNR gain of 0.5 to 3.0 dB, narrowing the gap between self-supervised and fully supervised models.
- Blind Denoising: The framework allows for "on-the-fly" estimation of unknown noise parameters. By minimizing image quality metrics like Total Variation during inference, the model can adapt to unknown noise levels in seconds, a significant improvement over methods requiring hours of retraining.
- Real-World Application: On the Fluorescence Microscopy Dataset (FMD), which features complex mixed Poisson-Gaussian noise, the two-step Tweedie approach proved more robust than specialized self-supervised competitors.
Discussion and Limitations
While Noise2Score provides a powerful unified theory, it is currently constrained to noise distributions within the exponential family. While this covers the majority of practical use cases (Gaussian, Poisson, Gamma, Bernoulli), it may not directly apply to highly complex, non-parametric noise found in some raw sensor data. Furthermore, as with any high-performance denoiser, there is a risk of over-smoothing fine textures, which remains a critical consideration for medical applications where subtle details are diagnostically significant.
Conclusion
Noise2Score represents a significant step forward in self-supervised image restoration. By unifying existing methods under the umbrella of Tweedie’s formula and score estimation, it provides a mathematically grounded, flexible, and high-performing framework that eliminates the need for clean reference images across a wide variety of scientific and industrial applications.
Summary updated