Introduction
Single-Instance Deep Generative Priors (SIDGPs), such as Deep Image Prior (DIP) and Deep Decoder, have emerged as powerful tools for image reconstruction (IR) tasks like denoising, inpainting, and medical imaging. Unlike traditional deep learning models that require massive datasets, SIDGPs optimize a neural network to fit a single corrupted image. While effective, these models suffer from a critical limitation: they eventually overfit to noise, requiring precise "early stopping" to capture the optimal reconstruction.
The Overfitting Challenge
The success of SIDGPs relies on the "spectral bias" of convolutional neural networks (CNNs), which naturally learn clean, low-frequency image features before high-frequency noise. This creates a characteristic "bell curve" in reconstruction quality. As training progresses beyond the peak, the network begins to fit random noise, causing the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) to plummet.
In real-world applications, the "clean" ground-truth image is unavailable, making it impossible to know when the peak has been reached. Most existing implementations rely on ad-hoc guesses, which limits the reliability of SIDGPs in production environments.
The Self-Validation Mechanism
To solve this, we introduce Self-Validation, a principled method for automatic early stopping. The approach utilizes a "side-car" autoencoder (AE) that monitors the reconstruction process in real-time.
Collaborative Training
The system runs two processes simultaneously:
- The Primal Process: The standard SIDGP optimization (e.g., DIP).
- The Side-Car AE: A small deep autoencoder trained "online" using a sliding window of the most recent reconstructed images.
The core insight is that when the primal process is at its peak quality, the reconstructed images are consistent and reside on a "natural image manifold," which the AE can easily learn to compress and reconstruct. When overfitting begins, the images become erratic and "un-natural," causing a sharp spike in the AE's reconstruction error.
By monitoring the valley of the AE's loss curve, the system can signal the optimal stopping point without ever seeing the ground-truth image.
Key Results and Versatility
The Self-Validation method was evaluated across standard denoising benchmarks, the ImageNet-100 dataset, and real-world MRI data. Key findings include:
- Near-Peak Performance: The "Detection Gap"—the difference between the true peak PSNR and the automatically stopped point—was typically less than 1 dB.
- Architecture Agnostic: The method works effectively across different architectures (DIP, Deep Decoder, SIREN) and various noise types (Gaussian, Impulse, Speckle).
- Superiority to Blind Metrics: Self-Validation significantly outperformed established no-reference image quality metrics like BRISQUE and NIQE, which often fail to identify the peak in generative contexts.
- Efficiency: The AE adds minimal computational overhead, as it is updated only once per training step in an online fashion.
Limitations
While robust, the method faces challenges with extremely high-texture images (e.g., the "Baboon" benchmark), where the AE may struggle to distinguish between complex natural textures and high-frequency noise. Additionally, in perfectly clean scenarios with no noise, the AE error curve can fluctuate, suggesting that future iterations could benefit from additional smoothing techniques.
Conclusion
Self-Validation provides a reliable, automated solution for the early stopping problem in SIDGPs. By treating the optimization history as a validation set, this method enables the deployment of deep generative priors in critical fields like medical imaging where ground-truth data is unavailable.
Summary updated