Abstract
Stochastic gradient Markov chain Monte Carlo (SG-MCMC) has become increasingly popular for simulating posterior samples in large-scale Bayesian modeling. However, existing SG-MCMC schemes are not tailored to any specific probabilistic model, even a simple modification of the underlying dynamical system requires significant physical intuition. This paper presents the first meta-learning algorithm that allows automated design for the underlying continuous dynamics of an SG-MCMC sampler. The learned sampler generalizes Hamiltonian dynamics with state-dependent drift and diffusion, enabling fast traversal and efficient exploration of neural network energy landscapes. Experiments validate the proposed approach on both Bayesian fully connected neural network and Bayesian recurrent neural network tasks, showing that the learned sampler out-performs generic, hand-designed SG-MCMC algorithms, and generalizes to different datasets and larger architectures.
Introduction
In Bayesian Deep Learning, the goal is to characterize the posterior distribution of network weights to quantify uncertainty. Stochastic Gradient MCMC (SG-MCMC) methods are the standard tools for this task, simulating particles moving through an "energy landscape" defined by the loss function. Traditionally, these samplers rely on hand-crafted physical dynamics derived from Hamiltonian mechanics or thermodynamics. However, designing these rules is mathematically intensive and often results in generic samplers that are inefficient for the high-dimensional, non-convex landscapes of deep neural networks.
Methodology: Learning the Physics of Sampling
The authors propose a meta-learning framework to automatically design the optimal dynamics for SG-MCMC. Following the "complete recipe" for SG-MCMC, any valid sampler can be defined by two state-dependent matrices: $D(z)$, representing diffusion and friction, and $Q(z)$, representing curl and acceleration.
Instead of manual tuning, these matrices are parameterized by small neural networks ($f_{\phi_D}$ and $f_{\phi_Q}$). These meta-learners take the current energy (loss), momentum, and local gradient as inputs. To train them, the authors optimize two novel objectives:
- Cross-chain loss: Minimizes the distance to the target distribution to accelerate convergence.
- In-chain loss: Maximizes the diversity of samples to ensure thorough exploration.
To overcome the intractability of the sampler's path probability density, the framework employs the Stein Gradient Estimator to approximate the gradients required for meta-training.
Experimental Results
The proposed "Neural Network SG-MCMC" (NNSGHMC) was evaluated against standard algorithms like SGHMC and SGLD across various tasks.
Synthetic and MNIST Benchmarks
On synthetic Gaussian distributions, NNSGHMC significantly outperformed standard SGHMC, achieving an Effective Sample Size (ESS) of 59 compared to 22. Visual analysis of the trajectories shows that NNSGHMC converges faster and explores the target distribution more effectively.
On the MNIST dataset, NNSGHMC demonstrated faster convergence and achieved superior Negative Log-Likelihood (NLL) scores compared to hand-designed baselines, while maintaining high classification accuracy (~98.36%).
Generalization and Transfer Learning
A key strength of the learned sampler is its ability to generalize. A sampler meta-trained on a simple 1-hidden-layer network can be effectively transferred to larger architectures (e.g., 2-hidden-layer networks) or different datasets. This suggests that the learned dynamics capture fundamental properties of neural network energy landscapes.
Insights into Learned Dynamics
Visualizing the learned $D(z)$ and $Q(z)$ functions reveals that the meta-learner discovers intuitive physical strategies. It learns to apply high acceleration in low-density regions to traverse them quickly and increases friction in high-density regions to stabilize sampling and prevent overshooting the target. Crucially, because the framework is built on a mathematically complete SDE framework, the sampler is guaranteed to converge to the true posterior in the limit of small step sizes.
Conclusion and Future Work
This work demonstrates that meta-learning can replace manual design in SG-MCMC, leading to samplers that are both faster and more robust. While the current approach involves significant computational overhead during meta-training and is limited to continuous Markov processes, it opens the door for more sophisticated samplers that incorporate "memory" features like moving averages of gradients, similar to modern optimizers like RMSprop.
Summary updated