Bayesian Inference through Stochastic Neural Sampling

This direction is split into other directions: neural sampling. Its last version stays readable and citable here.

Version 1 of 154 papers84 conceptsfrozen 2026-10-04

For decades, neuroscientists viewed the high variability of cortical neurons as a problem to be solved. If you show a monkey the same image twice, its neurons will fire differently each time. This "noise" was traditionally seen as a biological nuisance that the brain had to average out to achieve reliable perception.

However, a growing body of research suggests that this variability is not a bug, but a fundamental feature. In this view, the brain does not represent the world as a single "best guess" (a point estimate). Instead, it represents the world as a probability distribution. The fluctuating activity of neurons represents the brain performing Monte Carlo sampling—drawing random samples from an internal model to navigate an uncertain world.

The Sampling Hypothesis

The core idea, often called the neural sampling hypothesis, posits that the state of a neural network at any given moment is a single sample from a posterior probability distribution. Over time, the temporal statistics of these samples—how often certain neurons fire together or how activity fluctuates—reconstruct the full distribution of the brain's "beliefs."

Aapo Hyvarinen and Patrik O. Hoyer first proposed that the Poisson-like variability of firing rates is the signature of the brain performing Monte Carlo sampling to handle uncertainty. In this framework, the "noise" is actually the engine of computation, allowing the brain to explore different interpretations of ambiguous sensory data.

Recurrent Circuits as Internal Models

If neural activity represents samples, what determines the shape of the distribution they are sampled from? Research by Wenhao Zhang, Si Wu, and Brent Doiron suggests that the brain's recurrent connections store an "internal model" of the world's statistical structure.

In these models:

  1. Feedforward inputs represent the "likelihood" (the evidence from the senses).
  2. Recurrent connections represent the "prior" (what the brain expects based on experience).
  3. Intrinsic noise (like Poisson spiking) drives the system to explore the state space, effectively implementing a biological version of Gibbs sampling.

This explains why sparseness actually decreases with experience, as shown by Pietro Berkes and József Fiser. As a brain learns the complex statistics of its environment, its internal model becomes richer and more interconnected, leading to more complex, less sparse activity patterns that better reflect the true probability landscape of the world.

Efficient Exploration via Physics and Chaos

Sampling from high-dimensional distributions is computationally expensive. To speed this up, the brain may use more sophisticated algorithms than simple random walks. Laurence Aitchison and Máté Lengyel proposed that the brain implements Hamiltonian Monte Carlo (HMC). By mapping the "momentum" of HMC onto inhibitory neurons and the "position" onto excitatory neurons, the brain can use its natural oscillatory dynamics to explore probability landscapes much faster than stochastic noise alone would allow.

In some cases, the "noise" might not even be random. Taro Toyoizumi and Yu Terada have shown that recurrent networks can become chaotic, using deterministic variability to perform Bayesian sampling without needing any external source of randomness.

Explaining Perceptual "Flips" and Decision Making

The sampling hypothesis provides a natural explanation for perceptual bistability—why an ambiguous image like the Necker cube seems to "flip" between two interpretations. If the brain is sampling from a multimodal distribution (one with two peaks), it will occasionally jump from one peak to the other.

David P. Reichert and Amos Storkey demonstrated that biological "fatigue" (neuronal adaptation) actually helps this process. By temporarily weakening active connections, adaptation forces the sampler to leave its current peak and explore other high-probability interpretations, preventing the brain from getting "stuck" in one belief.

This sampling logic extends to behavior. When faced with the explore-exploit dilemma, the brain may use stochastic sampling to decide. As Momchil S. Tomov found, neural variance in motor areas scales with total uncertainty, suggesting that we literally "sample" possible actions based on their predicted probability of success.

From Biology to Hardware

This research has profound implications for neuromorphic engineering. If the brain's "bugs" (noisy spikes, unreliable synapses) are actually its computational "features," we can build more efficient AI by embracing them. Emre Neftci and colleagues have developed Synaptic Sampling Machines that use unreliable connections to perform low-power probabilistic inference, proving that the principles of neural sampling can be translated into robust, energy-efficient hardware.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_bayesian_inference_through_stochastic_ne_74770547,
  title = {Bayesian Inference through Stochastic Neural Sampling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/bayesian-inference-through-stochastic-neural-sampling/rdr_747705477eb94438af178bcd134081af},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_bayesian_inference_through_stochastic_ne_74770547_v1,
  title = {Bayesian Inference through Stochastic Neural Sampling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/bayesian-inference-through-stochastic-neural-sampling/rdr_747705477eb94438af178bcd134081af/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}