Abstract
We study the behavior of deterministic methods for solving inverse problems in imaging. These methods are commonly designed to achieve two goals: (1) attaining high perceptual quality, and (2) generating reconstructions that are consistent with the measurements. We provide a rigorous proof that the better a predictor satisfies these two requirements, the larger its Lipschitz constant must be, regardless of the nature of the degradation involved. In particular, to approach perfect perceptual quality and perfect consistency, the Lipschitz constant of the model must grow to infinity. This implies that such methods are necessarily more susceptible to adversarial attacks. We demonstrate our theory on single image super-resolution algorithms, addressing both noisy and noiseless settings. We also show how this undesired behavior can be leveraged to explore the posterior distribution, thereby allowing the deterministic model to imitate stochastic methods.
Introduction
In the field of image restoration, the quest for realism often comes at a hidden cost. Modern deterministic models, such as Conditional GANs (CGANs), have set new benchmarks for perceptual quality—producing images that are indistinguishable from natural ones—while maintaining consistency with the original degraded input. However, these high-performing models frequently exhibit a troubling instability: minute changes in input noise or blur can trigger drastic, unintended changes in the restored output.
This research moves beyond anecdotal observations of model "erraticness" to establish a fundamental mathematical principle. We prove that for any deterministic system, there is an inescapable tradeoff between achieving high perceptual quality and maintaining robustness to input perturbations.
The Lipschitz Explosion
The core of our theoretical contribution is the formalization of this tradeoff through the lens of Lipschitz continuity. A model's Lipschitz constant serves as a measure of its sensitivity; a low constant implies robustness, while a high constant indicates that small input changes can lead to large output variations.
Our primary theorem (Theorem 4.1) demonstrates that the Lipschitz constant of a deterministic restorer is lower-bounded by a factor inversely proportional to its "joint perceptual quality." We define this quality using the Wasserstein distance (or Earth Mover's Distance) between the joint distribution of the (restored image, input) and the (natural image, degraded input). As a model approaches "perfect" perceptual quality and consistency, its Lipschitz constant must mathematically explode to infinity. This instability is not a flaw of specific architectures but an inherent property of deterministic mapping in ill-posed inverse problems.
Empirical Evidence and Adversarial Vulnerability
To validate this theory, we evaluated over a dozen state-of-the-art super-resolution algorithms, including RealESRGAN, BSRGAN, and SwinIR. By measuring the Joint Earth Mover's Distance (JEMD) against the estimated Lipschitz lower bound ($\bar{K}$), we observed a clear inverse relationship across all models and degradation types.
This "Lipschitz explosion" translates directly into adversarial vulnerability. In face restoration tasks, high-quality models like GFPGAN proved remarkably easy to manipulate. By perturbing input pixels by as little as 1/255—a change invisible to the human eye—an adversary could flip the perceived gender of a restored face in over 70% of cases. In contrast, more stable models like RRDB, which produce blurrier but more robust outputs, remained significantly harder to fool.
Turning a Bug into a Feature: Posterior Exploration
Interestingly, the inherent instability of high-quality deterministic models can be repurposed. Because these models are forced to map a single input to a single "real-looking" output from a distribution of many possible valid solutions (the posterior), they become hyper-sensitive to input fluctuations.
We demonstrate that by slightly perturbing the input of an "erratic" model, one can effectively "sample" from the posterior distribution. Using a "Farthest Point Sampling" algorithm, we show that a single deterministic model can generate a diverse set of realistic variations for the same low-resolution input, mimicking the behavior of more complex stochastic diffusion models.
Conclusion
Our work establishes a "no free lunch" theorem for deterministic image restoration. The pursuit of perfect perceptual fidelity necessitates a loss of robustness, making models inherently susceptible to adversarial attacks. Understanding this tradeoff is crucial for the deployment of AI in sensitive imaging domains and opens new avenues for using deterministic instability as a tool for uncertainty estimation and posterior exploration.
Summary updated