Robust Stochastic Gradient Posterior Sampling with Lattice Based Discretisation
Lists
Only you see this
Abstract
Stochastic-gradient MCMC methods enable scalable Bayesian posterior sampling but often suffer from sensitivity to minibatch size and gradient noise. To address this, we propose Stochastic Gradient Lattice Random Walk (SGLRW), an extension of the Lattice Random Walk discretisation. Unlike conventional Stochastic Gradient Langevin Dynamics (SGLD), SGLRW introduces stochastic noise only through the off-diagonal elements of the update covariance; this yields greater robustness to minibatch size while retaining asymptotic correctness. Furthermore, as a comparison we analyse a natural analogue of SGLD utilising gradient clipping. Experimental validation on Bayesian regression and classification demonstrates that SGLRW remains stable in regimes where SGLD fails, including in the presence of heavy-tailed gradient noise, and matches or improves predictive performance.
BibTeX
@inproceedings{mensch2026robust,
author = {Zier Mensch and Lars Holdijk and Samuel Duffield and Maxwell Aifer and Patrick J. Coles and Max Welling and Miranda C. N. Cheng},
title = {{Robust Stochastic Gradient Posterior Sampling with Lattice Based Discretisation}},
booktitle = {International Conference on Machine Learning},
year = {2026},
url = {https://openreview.net/forum?id=mtRgrp0u7Z},
}Loading related content...