The ODE method for stability of skip-free Markov chains with applications to MCMC
Lists
Only you see this
Abstract
Fluid limit techniques have become a central tool to analyze queueing networks over the last decade, with applications to performance analysis, simulation and optimization. In this paper, some of these techniques are extended to a general class of skip-free Markov chains. As in the case of queueing models, a fluid approximation is obtained by scaling time, space and the initial condition by a large constant. The resulting fluid limit is the solution of an ordinary differential equation (ODE) in “most” of the state space. Stability and finer ergodic properties for the stochastic model then follow from stability of the set of fluid limits. Moreover, similarly to the queueing context where fluid models are routinely used to design control policies, the structure of the limiting ODE in this general setting provides an understanding of the dynamics of the Markov chain. These results are illustrated through application to Markov chain Monte Carlo methods.
BibTeX
@article{fort2008ode,
author = {Gersende Fort and Sean Meyn and Éric Moulines and Pierre Priouret},
title = {{The ODE method for stability of skip-free Markov chains with applications to MCMC}},
journal = {The Annals of Applied Probability},
year = {2008},
doi = {10.1214/07-aap471},
}| Subgeometric ergodicity of Markov chains | Lecture Notes in Statistics · 2006 |
| Optimal scaling of the random walk Metropolis algorithm under L p mean differentiability | J. Appl. Probab · 2017 |
| Diffusion limits of the random walk Metropolis algorithm in high dimensions | The Annals of Applied Probability · 2012 |
| Subgeometric rates of convergence in Wasserstein distance for Markov\n chains | arXiv · 2014 |
| Computable Convergence Rates for Subgeometrically Ergodic Markov Chains | arXiv · 2005 |
| A calculus for Markov chain Monte Carlo: studying approximations in algorithms | arXiv · 2023 |