Introduction
In the real world, making a prediction—such as estimating the amount of food at a specific location—is only half the battle. A survival-oriented brain must also know how certain that prediction is. The "Bayesian brain" hypothesis suggests that humans and animals integrate uncertainty optimally. However, standard machine learning tools for this task, like Gaussian Processes (GPs), rely on complex matrix inversions that are biologically impossible for a network of neurons to perform directly.
This paper proposes a biologically plausible neural network architecture that implements Gaussian Process regression, a powerful Bayesian framework for nonlinear prediction. By deriving local, Hebbian-style learning rules and using reinforcement-based updates for tuning curves, the author demonstrates how the brain could represent both the "average" expected outcome and the "risk" or uncertainty associated with it.
Bridging Bayesian Theory and Neurobiology
The core challenge addressed by this research is bridging the gap between high-level Bayesian theory and low-level neurobiology. Specifically, it asks: How can a network of neurons, using only local information available at each synapse, learn to perform non-parametric regression and represent predictive variance?
To solve this, the author maps the mathematical equations of Sparse Gaussian Process Regression (SGPR) onto a specific two-layer neural network.
Architecture and Learning Rules
- First Layer (Inducing Points): Neurons in this layer act as "inducing points," similar to hippocampal place cells. Their tuning curves correspond to the GP kernel (e.g., Radial Basis Functions).
- Output Layer: This layer features two distinct types of neurons. One encodes the predictive mean (average value), and the other encodes the predictive variance (uncertainty or risk).
- Local Weight Updates: Instead of relying on global backpropagation, the author derives local (anti-)Hebbian rules. The weight updates depend entirely on presynaptic activity, postsynaptic error, and a simple weight decay term.
- Hyperparameter Tuning: To update the centers and widths of the tuning curves (receptive fields), the model uses REINFORCE gradient estimates. This is biologically plausible as it mimics global neuromodulatory signals, such as dopamine, rather than complex error-routing.
To keep the mathematics "local," the author assumes that the neurons in the first layer are well-spread out. This allows for a diagonal approximation of the kernel matrix ($K_{uu}$), which significantly simplifies the learning circuitry.
Key Insights and Biological Plausibility
The paper suggests a direct functional mapping to specific brain regions. The first layer mirrors hippocampal place cells, while the output neurons mirror the orbitofrontal cortex (OFC), where separate neurons have been observed encoding reward value and reward risk.
Furthermore, the author proposes an "Up/Down" state mechanism. The brain could switch between representing total observation variance (including noise) and just the function's intrinsic uncertainty by transitioning between cortical "up" and "down" states. Ultimately, the complex matrix math of GPs is approximated by a simple least-squares problem that a neural network can solve iteratively through standard synaptic plasticity.
Key Results
The model, termed "BioNN" (Biologically plausible Neural Network), was evaluated on the classic 1D Snelson dataset and ten diverse regression datasets from the UCI Machine Learning Repository (including Boston Housing and Concrete Strength).
- Competitive Performance: BioNN achieved predictive accuracy (RMSE) and uncertainty calibration (NLPD) comparable to—and sometimes better than—standard ML methods like Probabilistic Back-propagation and Monte Carlo Dropout.
- Superior Noise Estimation: Unlike standard approximations (like VFE) that tend to over- or under-estimate noise, this neuronal model uses the actual target values ($y$) to calibrate its noise estimate. As a result, it was more accurate at inferring the true noise level of the data.
- Biological Realism: The model successfully replicated experimental observations, such as the accumulation of place fields near goal locations in spatial navigation tasks when tuning curves were optimized.
Limitations and Future Work
While promising, the model relies on the assumption that "inducing" neurons do not overlap too much (the diagonal $K_{uu}$ matrix approximation). If too many neurons are packed into the same space, this approximation might break down, though the author notes such dense packing is metabolically inefficient anyway.
Additionally, while the model matches the behavior of certain neurons in the OFC and hippocampus, direct experimental proof that these circuits are performing GP-like math is still needed. Finally, while tested on UCI datasets, the model's performance on extremely high-dimensional, complex tasks (like deep reinforcement learning) remains an open question for future research.
Summary updated