Training high-performance low-light denoisers typically requires a labor-intensive data acquisition process: capturing thousands of pairs of noisy short-exposure images and their perfectly aligned, clean long-exposure counterparts. This paper introduces a breakthrough noise synthesis method that bypasses this requirement, needing only two shots—one noisy image and one dark frame—to generate infinite, realistic training data that rivals the quality of massive real-world datasets.
The Challenge of Low-Light Noise
Deep learning-based denoisers are highly effective but depend heavily on the quality of training data. In low-light conditions, sensor noise is a complex mixture of signal-dependent components (photon shot noise) and signal-independent components (read noise, thermal noise, banding patterns). Standard models like Additive White Gaussian Noise (AWGN) fail to capture this complexity, while physics-based models often require tedious manual calibration of dozens of parameters. Generative models, while capable, typically require the very paired datasets this research aims to eliminate.
Methodology: 2-Shot Noise Synthesis
The proposed method decomposes noise modeling into two distinct components:
1. Signal-Dependent Noise
Photon shot noise is modeled using a Poisson distribution. The "gain" parameter, which determines how noise scales with light intensity, is estimated via linear regression from a single noisy image. A blurred version of this image serves as a "pseudo-clean" reference for the estimation.
2. Signal-Independent Noise (Spectral Sampling)
To model complex "read noise," the researchers utilize a single dark frame (captured with the lens cap on). Their core innovation is a Spectral Sampling algorithm that generates diverse noise realizations while preserving the sensor's unique spatial characteristics.
The algorithm involves:
- Fixed-Pattern Removal: Separating structured patterns from stochastic noise using Gaussian blur.
- Fourier-Domain Sampling: Computing the Fourier transform and randomizing the phase while preserving the magnitude spectrum. This maintains spatial correlations and "texture."
- Iterative Refinement: Alternating between Histogram Matching (to match statistical distributions) and Spectral Correction (to preserve spatial properties).
Key Insights
- Phase Randomization: Randomizing phase in the Fourier domain allows for infinite diversity in noise samples without losing the specific sensor's spatial signature.
- Inter-Channel Correlation (ICC): Replicating the same random phase across R, G, and B channels is essential. Treating channels independently leads to residual "banding" artifacts in the final denoiser.
- The Power of Iteration: Histogram matching alone destroys spatial structure, while spectral sampling alone misses statistical outliers. Iterating between the two creates synthetic noise indistinguishable from real data.
Experimental Evaluation
The method was evaluated on standard benchmarks including SID (Sony A7S2 subset), ELD, LLD, and LRID (testing generalization to smartphone sensors like the IMX686).
Performance Highlights
- Competitive Accuracy: On the SID dataset, the "2-shot" synthetic data model achieved a PSNR of 43.57 dB, nearly matching the 43.92 dB of a model trained on 1,865 real image pairs.
- Superior Visual Quality: The method consistently outperformed physics-based models like ELD and PMN, particularly in removing banding noise and color distortions.
- Generalization: The approach proved robust across different sensors and exposure ratios, demonstrating its practical utility for diverse hardware.
Conclusion
By leveraging Fourier-domain spectral sampling and minimal calibration data, this approach significantly reduces the barrier to training high-quality low-light denoisers. While future work may extend the model to account for thermal variables like sensor temperature, the current "2-shot" method provides a powerful, efficient alternative to labor-intensive data collection.
Summary updated