Abstract
There is a growing interest in deep model-based architectures (DMBAs) for solving imaging inverse problems by combining physical measurement models and learned image priors specified using convolutional neural nets (CNNs). For example, well-known frameworks for systematically designing DMBAs include plug-and-play priors (PnP), deep unfolding (DU), and deep equilibrium models (DEQ). While the empirical performance and theoretical properties of DMBAs have been widely investigated, the existing work in the area has primarily focused on their performance when the desired image prior is known exactly. This work addresses the gap in the prior work by providing new theoretical and numerical insights into DMBAs under mismatched CNN priors. Mismatched priors arise naturally when there is a distribution shift between training and testing data, for example, due to test images being from a different distribution than images used for training the CNN prior. They also arise when the CNN prior used for inference is an approximation of some desired statistical estimator (MAP or MMSE). Our theoretical analysis provides explicit error bounds on the solution due to the mismatched CNN priors under a set of clearly specified assumptions. Our numerical results compare the empirical performance of DMBAs under realistic distribution shifts and approximate statistical estimators.
Introduction
In computational imaging, solving "inverse problems"—reconstructing high-quality images from noisy, blurry, or incomplete measurements—is a fundamental challenge. Traditionally, these problems were addressed using hand-crafted mathematical priors. Recently, Deep Model-Based Architectures (DMBAs) have revolutionized the field by replacing these rules with learned Convolutional Neural Networks (CNNs). However, a critical question remains: how robust are these models when the CNN prior is "mismatched" to the test data? This occurs frequently in practice, such as when a model trained on natural images is applied to medical MRI scans, or when a CNN is used as a computationally efficient approximation of a complex statistical estimator.
Theoretical Framework: Quantifying the Mismatch
This research provides the first rigorous theoretical analysis of DMBAs under mismatched priors, focusing on the SD-RED (Steepest Descent variant of Regularization by Denoising) iteration. By utilizing monotone operator theory, the authors derived explicit error bounds that characterize how the reconstruction error scales with the degree of mismatch.
A central component of this analysis is the mismatch parameter, $\epsilon$, which quantifies the distance between the "true" desired prior operator $D_\sigma$ and the "mismatched" CNN prior $\hat{D}_\sigma$ actually employed during inference. The theory predicts that the final reconstruction error is proportional to this mismatch, ensuring that the algorithm remains stable rather than diverging when faced with out-of-distribution data.
As illustrated in the figure above, the distance between the operators increases with the noise level $\sigma$, confirming that the regularization strength directly influences the impact of the prior mismatch. This relationship bridges abstract operator theory with the practical behavior of neural networks in noisy environments.
Experimental Validation and Robustness
The study evaluated DMBAs across diverse datasets, including natural images (Set11), brain MRI (IXI dataset), and the fastMRI benchmark. The results highlight a surprising degree of resilience in deep priors:
- Cross-Domain Performance: In a 10% subsampled MRI reconstruction task, a prior trained on MRI data achieved 32.39 dB PSNR. Remarkably, a prior trained on natural images—a severe mismatch—still achieved 31.08 dB. Both significantly outperformed traditional hand-crafted methods like Total Variation (TV), which reached only 28.89 dB.
- CNNs as Approximators: The authors demonstrated that CNNs can be trained to approximate traditional TV priors. These "TV-Approx" priors yielded results nearly identical to exact mathematical solvers, suggesting that deep learning can accelerate classical algorithms without sacrificing accuracy.
- Sampling Resilience: The experiments showed that as the sampling ratio increases (providing more physical measurement data), the reconstruction becomes less sensitive to distribution shifts in the learned prior.
Conclusion
This work establishes that DMBAs are remarkably robust to mismatched priors. The theoretical error bounds and empirical evidence suggest that learned priors capture universal low-level features—such as edges and textures—that remain effective even across significant distribution shifts. While future work may explore more nuanced ways to characterize mismatch using Optimal Transport or test-time adaptation, these findings provide a strong theoretical foundation for the deployment of deep model-based architectures in real-world, imperfect scenarios.
Summary updated