Introduction: The Problem of Non-Smooth Potentials
Sampling from a probability distribution $p(x) \propto e^{-F(x)}$ is a cornerstone of modern statistics and machine learning, particularly in Bayesian inference. When the potential function $F(x)$ is smooth, the Unadjusted Langevin Algorithm (ULA) provides an efficient path to generate samples by following the gradient of the potential while incorporating Gaussian noise.
However, many critical applications involve non-smooth potentials. In Bayesian imaging, for instance, researchers frequently employ Total Variation (TV) penalties to promote edge-preserving smoothness or hard constraints to enforce physical plausibility (e.g., pixel values must be non-negative). In these scenarios, the gradient $\nabla F$ is undefined at certain points, rendering standard ULA inapplicable.
The Limitations of Moreau–Yoshida Smoothing
The conventional approach to handling non-smoothness is to replace $F$ with a smooth approximation, most commonly the Moreau–Yoshida (MY) envelope. While the MY envelope enables the use of gradient-based sampling, it introduces a significant structural flaw: it shifts the modes of the distribution.
The Maximum A Posteriori (MAP) estimator—the point where the distribution is most likely—is altered under MY smoothing. This creates a persistent bias in the resulting samples that does not vanish unless the smoothing parameter is set to an infinitesimally small value, which in turn leads to numerical instability and slow convergence.
Methodology: The Forward-Backward (FB) Envelope
This paper proposes a superior alternative: the Forward–Backward (FB) envelope. Originally developed for non-smooth optimization, the FB envelope is designed to smooth a composite function $F = f + g$, where $f$ is smooth and $g$ is non-smooth.
The critical advantage of the FB envelope is its mode-preserving property. For a wide range of smoothing parameters, the minimum points (modes) of the FB envelope are identical to those of the original function $F$. By sampling from a distribution defined by the FB envelope, the resulting Markov chain spends more time in the true high-probability regions of the target distribution.
The EULA Framework
To operationalize this, the authors introduce the Envelope Unadjusted Langevin Algorithm (EULA). This general framework allows for the integration of any "admissible" smooth envelope—including both MY and FB—into a Langevin-style discretization. This provides a unified theoretical bridge between optimization techniques and Markov Chain Monte Carlo (MCMC) sampling.
Theoretical Guarantees and Convergence
The authors provide a rigorous non-asymptotic analysis of the EULA framework. They establish bounds on the number of iterations required to reach a target accuracy, measured by the 1-Wasserstein metric. A key theoretical finding is that the FB envelope exhibits strong convexity when far from the center of the distribution, which ensures reliable convergence even in high-dimensional spaces.
Empirical Evaluation
The method was validated across two distinct domains:
- Truncated Gaussians: In synthetic tests ranging from 2 to 100 dimensions, the FB-based algorithm (FBULA) significantly outperformed the Moreau–Yoshida version (MYULA). As the dimensionality increased, the bias inherent in the MY envelope became more pronounced, whereas FBULA maintained high accuracy in estimating mean and variance.
- Tomographic Image Reconstruction: Using the standard Shepp–Logan phantom, the authors demonstrated that FBULA achieves a lower Mean Squared Error (MSE) in reconstructing images from noisy Fourier measurements. The resulting images were visually sharper and more faithful to the ground truth compared to those generated via MYULA.
Conclusion and Future Outlook
The introduction of the Forward-Backward envelope to Langevin sampling represents a significant step forward for non-smooth Bayesian inference. By preserving the modes of the target distribution, FBULA offers a more accurate and robust sampling mechanism than traditional smoothing techniques.
While the method requires higher-order smoothness ($C^3$) for the differentiable component $f$ and involves second-derivative calculations, its performance gains in high-dimensional, constrained problems are substantial. Future work may explore the application of FB envelopes to non-convex potentials, potentially unlocking new capabilities in complex sampling tasks.
Summary updated