In classical imaging science, solving an "inverse problem"—like reconstructing an MRI scan from raw data or deblurring a photo—usually involves minimizing a cost function. This function balances two things: how well the solution fits the observed data and how well it matches our prior expectations of what a "good" image looks like.
For decades, these priors were handcrafted mathematical formulas, such as Total Variation. Recently, researchers have shifted toward Plug-and-Play (PnP) and Regularization by Denoising (RED) frameworks. As Hong Ye Tan, Subhadip Mukherjee, and Junqi Tang explain in From Image Denoisers to Regularizing Imaging Inverse Problems: An Overview, these methods allow us to "plug" a pre-trained deep learning denoiser directly into an iterative optimization loop.
The challenge is that neural networks are often "black boxes." If a denoiser is slightly too aggressive or unpredictable, the iterative loop can oscillate or diverge entirely, producing artifacts instead of a clean image. To fix this, a growing body of research focuses on enforcing mathematical "guardrails" on these networks to guarantee they behave predictably.
The Intuition of Non-Expansiveness
The core requirement for stability in an iterative system is that the operator must not "blow up" the signal. In mathematical terms, we want the denoiser to be non-expansive.
Imagine an iterative process as a ball rolling down a hill toward a valley (the solution). If the denoiser is non-expansive, it ensures that every time we apply it, we aren't kicking the ball so hard that it flies out of the valley. As noted by Lei Tian and Yu Sun in their work on optical tomography, enforcing this constraint prevents the denoiser from amplifying signal energy, which is critical for the algorithm to settle on a stable solution.
Enforcing Stability via Spectral Normalization
How do we force a complex neural network to be non-expansive? The most common tool is spectral normalization.
A network is non-expansive if its Lipschitz constant is less than or equal to 1. This means the output change is never greater than the input change. Ernest K. Ryu and colleagues provided a theoretical breakthrough in Plug-and-Play Methods Provably Converge with Properly Trained Denoisers by introducing real spectral normalization (realSN). By constraining the weights of the network during training, they proved that PnP algorithms would converge to a fixed point rather than diverging.
This concept has been extended across various architectures:
- Multispectral Imaging: Jiaming Liu and Ulugbek S. Kamilov used spectral normalization in MS-RED to ensure that 3D denoisers act as stable regularizers.
- Deep Equilibrium Models: In ODER, Jiaming Liu and Weijie Gan applied Lipschitz constraints to ensure that "unrolled" networks converge to a unique stable state efficiently.
- Bilevel Optimization: Risheng Liu and Shangzhi Zeng demonstrated in Optimization-Derived Learning that these constraints are essential for maintaining stability even when the model encounters data it wasn't specifically trained on.
Balancing Stability and Performance
A common critique of strict Lipschitz constraints is that they can limit a network's expressive power, potentially "killing" its performance. To address this, Pakshal Bohra and Michael Unser proposed deep spline neural networks. By combining spectral normalization of weights with slope normalization of learnable activation functions, they maintained the 1-Lipschitz requirement without sacrificing the high-quality reconstruction that deep learning is known for.
Beyond Point Estimates: Bayesian Stability
The need for Lipschitz control extends beyond simple image reconstruction into the realm of uncertainty. When using PnP priors for Bayesian sampling (trying to understand the range of possible solutions rather than just one), stability becomes even more fragile.
In Efficient Bayesian Computation Using Plug-and-Play Priors, Teresa Klatzer and Marcelo Pereyra show that controlling the spectral norm of the denoiser's Jacobian is vital. Without this regularity, the long Markov chains used in sampling can become numerically unstable, leading to "exploding gradients" and nonsensical results.
By treating neural networks as mathematically bounded operators rather than unconstrained black boxes, this research direction transforms deep learning from a powerful but risky heuristic into a reliable, provable component of scientific imaging.