Abstract: We establish $L_q$ convergence for Hamiltonian Monte Carlo algorithms. More specifically, under mild conditions for the associated Hamiltonian motion, we show that the outputs of the algorithms converge (strongly for $2\le q < \infty$ and weakly for $1 < q < 2$) to the target distribution. Our results provide a theoretical foundation for the use of Hamiltonian Monte Carlo in a wide range of applications.
Introduction
Hamiltonian Monte Carlo (HMC) has emerged as a "gold standard" for generating random samples from complex, high-dimensional probability distributions, particularly in Bayesian inference and machine learning. While HMC is used extensively in practice, theoretical proofs of its convergence have historically been limited to specific settings, such as $L^2$ Hilbert spaces. While mathematically convenient, $L^2$ spaces do not cover the full spectrum of error metrics and robustness requirements encountered in modern statistical modeling.
This research provides a rigorous theoretical foundation for HMC by proving its convergence within a broader class of functional spaces known as $L^q$ spaces. By extending the mathematical understanding of HMC to these spaces, the authors provide a more flexible framework for analyzing the algorithm's performance across diverse machine learning applications.
Theoretical Framework: HMC as a Linear Operator
The paper approaches HMC through the lens of functional analysis. Rather than analyzing individual sample trajectories, the authors treat the HMC iteration as a linear operator, denoted as $\mathcal{T}$, acting on a space of integrable functions (probability densities). This shift in perspective allows for the application of operator theory to establish global convergence properties.
Key Methodological Components
- Invariance and the Covering Property: The authors establish "mild conditions" for the Hamiltonian motion, including measure invariance and a "covering property." The latter ensures that the Hamiltonian motion eventually explores the entire state space, a prerequisite for irreducibility.
- The Averaging Map: A core insight of the paper is that the HMC iteration functions as an "averaging map." By adding auxiliary momentum and subsequently projecting back, the algorithm naturally smooths out distributions toward the target density.
- Convexity and Contraction: Using the mathematical property of convexity (specifically for the function $x^q$), the authors demonstrate that the HMC operator is a "contraction." This means that applying the HMC step never increases the distance between the current distribution and the target distribution.
- Self-Adjointness: The authors identify conditions under which the HMC operator is self-adjoint (mathematically symmetric). This property is crucial for simplifying the proof of convergence and identifying the target distribution as the unique stable state.
Convergence Results
The primary contribution of the paper is the formal proof that the sequence of distributions generated by HMC is guaranteed to converge to the target distribution $f$. The nature of this convergence depends on the value of $q$:
- Strong Convergence ($2 \le q < \infty$): For these values, the authors prove that the actual distance between the algorithm's output and the target distribution goes to zero in the $L^q$ norm.
- Weak Convergence ($1 < q < 2$): In this range, the algorithm's outputs settle into the target distribution in a more generalized, weak sense.
In both cases, the target distribution is shown to be the unique "eigenvalue 1" of the HMC operator, meaning it is the only fixed point the algorithm can reach.
Discussion and Limitations
While this work provides a significant leap in the theoretical understanding of HMC, several areas remain for future exploration:
- Self-Adjointness Requirements: The results rely on the operator being self-adjoint. While the authors provide a sufficient condition using a measure-preserving involution, not all practical implementations of HMC may satisfy this requirement.
- Convergence Rates: The paper proves that the algorithm converges, but it does not explicitly calculate how fast it does so (the convergence rate) in $L^q$ spaces. The authors suggest that under specific conditions, this convergence could be exponential.
- Irreducibility Assumptions: The "covering property" is a relatively strong assumption. Future technical refinement could involve weakening this to "eventual coverage," which would broaden the applicability of these proofs to even more complex Hamiltonian systems.
By establishing these $L^q$ convergence guarantees, this research reinforces the mathematical validity of HMC and provides a robust theoretical baseline for researchers developing new sampling techniques in non-standard functional spaces.
Summary updated