Gradient-free MCMC methods for dynamic causal modelling

Biswa Sengupta, Karl J. Friston, William D. PennyNeuroImage 2015 · DOI

TL;DR

This paper evaluates four gradient-free Markov Chain Monte Carlo (MCMC) sampling methods for dynamic causal modeling (DCM), a technique used in neuroimaging. The authors found that adaptive MCMC and population-based MCMC samplers are significantly more efficient than standard random walk Metropolis or slice sampling, with adaptive MCMC showing the most promise for computational speed.

What Problem Does This Paper Address?

Dynamic Causal Models (DCMs) are powerful tools for understanding brain connectivity by modeling neural activity. However, inferring the parameters of these complex models can be computationally challenging. Traditional methods like Laplace approximations can underestimate parameter variability and struggle with multi-modal posteriors. While Markov Chain Monte Carlo (MCMC) methods offer a more robust solution by sampling from the posterior distribution, they can be computationally expensive. This paper specifically addresses the trade-off between computational efficiency and sampling accuracy for gradient-free MCMC methods, aiming to identify the most suitable algorithms for DCM inference.

Methods

The authors compared four gradient-free MCMC sampling algorithms:

  • Random Walk Metropolis (RWM) Sampler: A standard MCMC algorithm where proposals are generated from a Gaussian distribution centered on the current state.
  • Slice Sampler: This method samples from the target distribution by introducing an auxiliary variable and sampling from a "slice" of the distribution. It doesn't require a proposal distribution.
  • Adaptive MCMC Sampler: This algorithm tunes the proposal distribution's mean and covariance based on the history of samples, aiming to improve convergence speed. It uses stochastic approximations to adapt these parameters.
  • Population MCMC Sampler: This approach involves running multiple Markov chains at different "temperatures" (inverse of the posterior density's sharpness) and allowing proposals to be swapped between chains. This helps to explore multi-modal posteriors and improve mixing.

The evaluation was performed using a single-node neural mass model (NMM) with known parameters to simulate synthetic data. Computational efficiency was benchmarked by measuring the time taken to produce independent samples.

Datasets

The paper does not use real-world datasets. Instead, it generates synthetic data by simulating a single-node neural mass model (NMM) with known parameters. This allows for a controlled evaluation of the MCMC algorithms' performance against a ground truth.

Key Insights and Ideas

  • Adaptive MCMC is a strong contender for speed: The adaptive MCMC sampler, which tunes its proposal distribution, significantly outperforms other methods in terms of computational time per independent sample. This is crucial for making MCMC practical for complex neuroimaging models.
  • Slice sampling offers high-quality samples but at a cost: While slice sampling produces the highest number of independent samples from the target density, it comes with a substantial increase in computational time, making it less practical for routine use.
  • Population MCMC aids in exploring complex posteriors: The population MCMC sampler, with its multi-chain approach and proposal swaps, is particularly useful for models with multi-modal posteriors, which are common in DCMs. It offers a good balance between ESS and computational time, especially when using a power-law temperature schedule.
  • The trade-off between mixing and compute time is central: The paper highlights the inherent trade-off in MCMC: rapid chain mixing (getting to the target distribution quickly) versus computational efficiency (using fewer processing cycles). The "best" algorithm depends on the specific priorities.

Key Results

  • Adaptive MCMC was the most computationally efficient, taking approximately 0.4-0.6 minutes to generate a single uncorrelated sample.
  • Population MCMC (power schedule) offered a good balance, with a slightly higher computational cost than adaptive MCMC but a better effective sample size (ESS) and a significant speedup compared to slice sampling.
  • Slice sampling yielded the highest ESS but was considerably slower, taking about 1000% more computational time than adaptive MCMC.
  • Random Walk Metropolis was the least efficient, exhibiting slow convergence and low ESS.
  • The paper's evaluation metric was the "time normalized ESS," which represents the time spent sampling a single uncorrelated sample, serving as a measure of worst-case behavior.

Limitations and Future Work

  • Limited model complexity: The evaluation was performed on a single-node NMM. Future work could extend this to more complex, multi-node DCMs with a larger number of parameters.
  • Gradient-free focus: This paper exclusively examines gradient-free methods. The authors suggest that future research could explore combining gradient-free MCMC estimators with gradient-based methods for even greater efficiency.
  • Expert intervention for Population MCMC: The population MCMC sampler requires careful tuning of parameters like the number of chains and temperature ladder, which necessitates expert intervention.
  • Convergence diagnostics: The paper briefly touches upon the challenge of determining when an MCMC chain has converged, a critical aspect for reliable inference. Further research into robust convergence diagnostics for these methods would be valuable.

Summary updated

BibTeX
@article{sengupta2015gradient,
  author = {Biswa Sengupta and Karl J. Friston and William D. Penny},
  title = {{Gradient-free MCMC methods for dynamic causal modelling}},
  journal = {NeuroImage},
  year = {2015},
  doi = {10.1016/j.neuroimage.2015.03.008},
}
Figure