Neurons as Monte Carlo Samplers: Bayesian Inference and Learning in Spiking Networks

Yanping Huang, Rajesh P. N. RaoNeurIPS 2014 · OpenReview

TL;DR

This paper proposes that the brain performs Bayesian inference by treating individual spikes as Monte Carlo samples of hidden world states. By utilizing stochastic synapses and a spike-timing-dependent Hebbian learning rule, the authors demonstrate how a spiking neural network can both track and learn the dynamics of complex environments (Hidden Markov Models) in real-time.

The Challenge of Neural Inference

Humans routinely estimate world states from noisy, ambiguous stimuli—like a tennis player predicting a ball's trajectory based on a fleeting glimpse. Mathematically, the optimal way to handle this uncertainty is through Bayesian inference, often modeled using Hidden Markov Models (HMMs).

However, a major gap exists between high-level Bayesian theory and low-level neurobiology. Previous models often relied on "firing rate" abstractions (which ignore the discrete nature of spikes) or assumed that the rules of the world (transition and emission probabilities) were already known to the brain. This paper tackles a fundamental question: How can a network of biological, spiking neurons perform both inference (tracking the state) and learning (figuring out the rules) simultaneously?

A Spiking Network for Monte Carlo Sampling

The authors introduce a two-layer spiking neural network model designed to bridge this gap:

  1. Sensory Layer: Detects noisy measurements of the world.
  2. Inference Layer: A recurrently connected population where neurons are grouped into "sub-populations" (analogous to cortical columns), each representing a specific preferred world state.
Network architecture and Monte Carlo sampling mechanism

Key Innovations

  • Spikes as Particles: Instead of a neuron's firing rate representing a probability value, a single spike is viewed as a Monte Carlo sample (a "particle") drawn from the posterior distribution.
  • Stochastic Synapses: Unlike traditional neural networks where weights are fixed scalars, this model treats synaptic weights as release probabilities. A spike only triggers a post-synaptic potential (EPSP) with a certain probability, providing the inherent randomness needed for sampling.
  • Coincidence Detection: An inference neuron only fires if it receives both a recurrent input (representing a "prediction" from the previous state) and a feedforward input (representing the current "measurement").
  • Online EM Learning: The authors derive a biologically plausible, spike-timing-dependent Hebbian learning rule that implements an online version of the Expectation-Maximization (EM) algorithm, allowing the network to learn transition and emission matrices from scratch.

Key Insights

  • Noise is a Feature, Not a Bug: The "Poisson-like" variability commonly observed in cortical responses is often dismissed as noise. This paper argues it is actually the necessary variability for sampling-based inference.
  • Synapses as Probabilistic Units: By modeling synapses as probabilistic release sites rather than simple scalars, the authors align the model with physiological evidence (e.g., the fact that only a few NMDA receptors might open per release event).
  • Local Learning of Global Dynamics: The complex math of the EM algorithm—usually done "off-line" with access to all data—can be reduced to local Hebbian updates that occur every time a neuron spikes.

Experimental Results

The model was evaluated using synthetic simulations of HMMs designed to test specific capabilities, including stochastic volatility models and bi-modal posterior tracking tasks.

Tracking performance and posterior distribution estimation
  • Consistent Estimation: The population spike count was shown to be a consistent estimator of the true posterior probability. As the number of neurons/spikes increases, the error approaches zero. The network successfully tracks hidden variables even in highly noisy environments.
  • Poisson Variability: The model naturally produced a "variance proportional to mean" relationship in neural responses, matching biological data.
  • Power-Law Convergence: The learning rules demonstrated robust convergence. For example, the Mean Square Error (MSE) for learning the emission matrix followed a power law of $k^{-1.1}$ (where $k$ is time). Even in high-noise scenarios, the network successfully learned the underlying transition dynamics of the environment.

Limitations and Future Work

  • Filtering vs. Smoothing: The model performs "filtering" (estimating the current state given past data). While computationally efficient and biologically relevant, it is theoretically less accurate than "smoothing" (estimating past states given all data).
  • Rigorous Proofs: While numerical simulations show convergence, the authors note that a rigorous mathematical proof of convergence for this specific spiking implementation of the EM algorithm remains an open challenge.
  • Scaling to High Dimensions: The experiments focused on relatively low-dimensional state spaces. Future work is needed to see how these sampling-based networks scale when dealing with the high-dimensional, multi-modal sensory data found in the real world.

Summary updated

Abstract

We propose a two-layer spiking network capable of performing approximate inference and learning for a hidden Markov model. The lower layer sensory neurons detect noisy measurements of hidden world states. The higher layer neurons with recurrent connections infer a posterior distribution over world states from spike trains generated by sensory neurons. We show how such a neuronal network with synaptic plasticity can implement a form of Bayesian inference similar to Monte Carlo methods such as particle filtering. Each spike in the population of inference neurons represents a sample of a particular hidden world state. The spiking activity across the neural population approximates the posterior distribution of hidden state. The model provides a functional explanation for the Poisson-like noise commonly observed in cortical responses. Uncertainties in spike times provide the necessary variability for sampling during inference. Unlike previous models, the hidden world state is not observed by the sensory neurons, and the temporal dynamics of the hidden state is unknown. We demonstrate how this network can sequentially learn the hidden Markov model using a spike-timing dependent Hebbian learning rule and achieve power-law convergence rates.
BibTeX
@inproceedings{huang2014neurons,
  author = {Yanping Huang and Rajesh P. N. Rao},
  title = {{Neurons as Monte Carlo Samplers: Bayesian Inference and Learning in Spiking Networks}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2014},
  url = {https://openreview.net/forum?id=ByWkAu-_Wr},
}
Figure