Abstract
SST-GP is a novel self-supervised image denoising framework that leverages Gaussian Processes (GP) to model the joint relationship between multiple noisy realizations of a single image. By generating pseudo-ground truth labels through GP and employing a dual-branch Denoising Transformer (Den-T), the method effectively captures both fine local details and broad global context. Experimental results on synthetic and real-world datasets demonstrate that SST-GP outperforms state-of-the-art self-supervised methods without requiring clean training data.
Introduction
Most state-of-the-art image denoising methods rely on supervised learning, which requires massive datasets of paired noisy and perfectly clean images. In real-world scenarios, such as low-light photography or medical imaging, obtaining a "clean" ground-truth version is often impossible. While self-supervised methods have emerged to learn from noisy data alone, they face significant hurdles in modeling complex real-world noise and are often limited by the narrow receptive fields of standard Convolutional Neural Networks (CNNs).
To address these challenges, we present SST-GP, which combines statistical modeling via Gaussian Processes with the long-range dependency modeling of Transformers. This approach eliminates the need for clean ground-truth images while providing a more robust architectural foundation for distinguishing signal from noise.
Methodology
The SST-GP framework introduces two primary innovations: a Gaussian Process-based self-supervision strategy and a specialized Denoising Transformer (Den-T) architecture.
Self-Supervision via Gaussian Processes
Instead of comparing a noisy image to a clean one, SST-GP generates multiple down-sampled versions of a single noisy image using random cyclic shifts. These versions share the same underlying clean signal but possess different noise realizations. We utilize a Gaussian Process—a non-parametric statistical model—to formulate a joint distribution between these versions. By learning the covariance between these images, the GP predicts a "pseudo-ground truth" for each version, allowing the network to learn noise characteristics directly from the data.
The Denoising Transformer (Den-T)
To overcome the local receptive field limitations of CNNs, we designed the Den-T architecture with two parallel branches:
- Fine Context Transformer Branch (FTB): This branch focuses on high spatial resolution to preserve intricate details and textures.
- Coarse Context Transformer Branch (CTB): This branch captures the global structure of the image, helping the model differentiate between large-scale features and random noise.
- Hybrid Design: Depth-wise convolutions are integrated into the Transformer's feed-forward layers to maintain spatial locality, ensuring the model retains pixel-level precision while processing global context.
Experiments and Results
We evaluated SST-GP on standard synthetic benchmarks (Kodak, BSD, Set-14) and the real-world Smartphone Image Denoising Dataset (SIDD).
Quantitative Performance
On synthetic Gaussian noise ($\sigma=25$), SST-GP achieved a PSNR of 32.75 dB on the Kodak dataset, significantly outperforming previous self-supervised methods like Neighbor2Neighbor (32.08 dB) and Noise2Void (30.32 dB). On the real-world SIDD benchmark, SST-GP reached 50.87 dB, surpassing traditional methods like BM3D and other self-supervised CNN-based approaches.
Qualitative Analysis
Visually, SST-GP produces sharper edges and fewer artifacts than its predecessors. By modeling the joint relationship of multiple image versions rather than simple pairs, the GP provides a more stable teaching signal, leading to cleaner reconstructions that closely approximate the ground truth.
Limitations and Future Work
The primary trade-off for SST-GP's performance is computational overhead. The Gaussian Process involves complex matrix operations, making SST-GP approximately 1.5x slower to train than standard L2-norm models. Additionally, the memory footprint is higher due to the storage of intermediate latent vectors. Future research will focus on optimizing GP computations and exploring more efficient kernel functions to scale the method for real-time applications and larger datasets.
Conclusion
SST-GP represents a significant step forward in self-supervised denoising. By combining the statistical rigor of Gaussian Processes with the structural flexibility of Transformers, it provides a powerful solution for high-quality image restoration in environments where clean data is unavailable.
Summary updated