Deep learning has revolutionized image denoising, but most state-of-the-art models are "supervised"—they require pairs of noisy and perfectly clean images for training. In fields like medical imaging or astronomy, clean ground-truth images are often impossible to obtain.
Existing "self-supervised" alternatives that train only on noisy data have significant drawbacks. For instance, Noise2Noise requires multiple noisy versions of the exact same image. Methods like Noise2Void and Noise2Self work on single images but can struggle with certain noise types or lose fine details. Meanwhile, SURE-based methods are mathematically elegant but highly sensitive to hyperparameters and must be completely retrained if the noise level or model changes.
Noise2Score is a novel self-supervised framework that unifies these disparate methods under a single Bayesian umbrella. By leveraging Tweedie’s formula, the authors demonstrate that denoising any exponential family noise (e.g., Gaussian, Poisson, Gamma) can be reduced to a single task: estimating the "score function" (the gradient of the log-likelihood) of the noisy data.
The Noise2Score Methodology
The core innovation of this paper is the application of Tweedie’s formula, a classical result from Bayesian statistics. It provides an explicit way to calculate the posterior mean (the "best guess" for the clean image) using only the noisy measurement and the score function of the noisy data distribution.
The Noise2Score pipeline breaks the denoising process into two distinct steps:
- Score Function Estimation: The authors use an Amortized Residual Denoising Autoencoder (AR-DAE) to estimate the score function from noisy images. Crucially, this training step is "universal"—the network learns the structure of the noise in the data without needing to know the specific noise model (Gaussian vs. Poisson) beforehand.
- Tweedie’s Post-processing: Once the score is estimated, the final denoised image is obtained by plugging that score into a specific version of Tweedie’s formula tailored to the noise type.
As shown in the comparison above, while standard methods try to map directly from noisy to clean images, Noise2Score first extracts the statistical "score" of the noise before applying a mathematical formula to retrieve the clean image.
Key Insights
- The Universal Denoiser: Because the score function estimation is independent of specific noise parameters, you do not need to retrain the neural network if the noise level changes. You simply adjust the parameters in the Tweedie formula during inference.
- Bridging the Gap: The paper mathematically demonstrates that the loss functions used in DAEs and SURE are essentially different ways of trying to find this same "score function." This provides a unified theoretical foundation for the field.
- Blind Denoising: Noise2Score excels at "blind" setups where the noise level (like the standard deviation $\sigma$ of Gaussian noise) is unknown. By using an image quality metric like Total Variation (TV) during the post-processing step, the optimal noise parameters can be found "on-the-fly" without retraining the model.
Experimental Results
The authors evaluated their method on standard computer vision benchmarks, including BSD68 and Set12 for grayscale, and CBSD68 and Kodak for color images. The models were trained using DIV2K and CBSD400 datasets with synthetic Gaussian, Poisson, and Gamma noise.
Noise2Score consistently outperformed other single-image self-supervised methods across all tested noise types. In Gaussian denoising ($\sigma=25$), Noise2Score achieved a PSNR of 29.12 dB on BSD68, significantly higher than Noise2Void (26.77 dB) and SURE (28.78 dB). Furthermore, the method retains significantly more texture and sharp edges compared to competitors, which often produce over-smoothed results. The performance gains were even more pronounced for non-Gaussian noise (Poisson and Gamma), where traditional methods like SURE are harder to adapt.
Limitations and Future Directions
While highly effective, the current derivation relies on the noise following an exponential family distribution. While this covers the vast majority of common noise types, it might not directly apply to more complex, structured "real-world" noise. Additionally, while the model doesn't need to know the noise parameters during training, the user still needs to know the type of noise (e.g., "this is Poisson noise") to choose the correct Tweedie formula for the final step.
Future work may extend this framework to handle non-exponential noise and explore its application in specific domains like low-dose CT scans or microscopy, where clean data is notoriously scarce.
Summary updated