Abstract
Deep learning has significantly advanced image denoising, yet most state-of-the-art models require massive datasets of clean and noisy image pairs. In specialized fields like cryo-electron microscopy (cryo-EM), such datasets are often unavailable. While self-supervised methods can learn from a single noisy image, they frequently suffer from overfitting, where the model memorizes specific noise patterns rather than learning generalizable image features. This paper proposes a novel self-supervised framework that utilizes Tucker low-rank tensor decomposition and a weight distortion scheme within an Alternating Direction Method of Multipliers (ADMM) framework to regularize the network, preventing overfitting and enhancing out-sample generalization.
Introduction
The "data tax" of supervised deep learning is a major bottleneck in scientific imaging. When clean ground-truth images are impossible to obtain, researchers turn to self-supervised methods like Deep Image Prior (DIP). However, these methods face a fundamental challenge: without a large variety of training data, the model's high capacity allows it to fit the noise itself. This leads to poor performance when the model is applied to images other than the one it was trained on.
This research introduces a method to boost generalization by periodically forcing the neural network's weights into a low-rank structure. By treating model compression—specifically Tucker decomposition—as a mathematical constraint during training, the authors prevent the network from "memorizing" noise, effectively forcing it to learn the underlying structural components of the image.
Methodology
1. Tucker Low-Rank Approximation
Convolutional neural networks (CNNs) utilize 4D weight tensors (height, width, input channels, and output channels). The authors apply Tucker decomposition to these tensors to exploit their latent low-rank structures. This reduces the degrees of freedom in the model, acting as a powerful regularizer.
2. Automatic Rank Selection (VBMF)
To avoid the trial-and-error process of manual rank selection, the framework incorporates Variational Bayesian Matrix Factorization (VBMF). This allows the model to automatically determine the optimal rank for each layer based on the specific characteristics of the data, ensuring the constraint is neither too loose (allowing noise memorization) nor too tight (losing image detail).
3. The Weight Distortion Scheme
The core innovation is a periodic "twist" or projection of weights onto a low-rank manifold every 200 iterations. This weight distortion helps the optimizer find "flatter" regions in the loss landscape. In deep learning theory, flat minima are associated with superior generalization because the model becomes less sensitive to small perturbations in the input data.
4. ADMM Framework
The denoising task is formulated as an optimization problem solved via ADMM, which splits the process into three sub-problems:
- Updating CNN parameters.
- Updating an auxiliary image variable (using a traditional denoiser like BM3D as a structural prior).
- Updating a dual variable to maintain consistency between the CNN output and the prior.
Experimental Results
Synthetic Benchmarks
The method was evaluated on standard datasets (SET12 and BSD68) with Gaussian and Poisson-Gaussian noise. On the SET12 dataset ($\sigma=25$), the proposed method achieved a PSNR of 30.44dB, notably outperforming the supervised DnCNN (29.55dB) and the standard unsupervised NN+BM3D (29.81dB).
Real-World Cryo-EM
The framework was applied to raw micrographs of the SARS-CoV-2 2P protein. These images are characterized by extremely low signal-to-noise ratios. The proposed model was not only effective but also highly efficient, performing inference roughly 5x faster than the traditional BM3D algorithm (300 seconds vs. 1500 seconds per micrograph).
Generalization and Robustness
In "out-sample" tests—where a model trained on one image (e.g., LENA) is tested on a different dataset (BSD68)—the low-rank approximated CNN significantly outperformed existing unsupervised methods. Sensitivity analysis further confirmed that the model remains stable across various hyperparameter configurations, making it a practical tool for non-experts.
Discussion and Limitations
While the low-rank constraint is highly effective for high-noise environments, the authors noted a minor quality trade-off in very low-noise scenarios ($\sigma=20$). In these cases, the aggressive regularization may filter out fine, legitimate image details. Additionally, while the VBMF rank selection works well for grayscale images, future work is needed to account for the high correlation between color channels in RGB images.
Conclusion
By integrating tensor decomposition directly into the optimization loop, this research demonstrates that "less is more." Restricting a network's capacity through low-rank approximations does not just compress the model; it serves as a vital regularizer that enables self-supervised models to generalize beyond their training samples, providing a robust solution for high-stakes imaging tasks where clean data is a luxury.
Summary updated