Self-supervised learning of probabilistic prediction through synaptic plasticity in apical dendrites: A normative model

A. Ravishankar Rao, Robert Legenstein, Anand Subramoney, Wolfgang MaassbioRxiv 2021 · DOI

TL;DR

This paper proposes a novel learning rule, Dendritic Logistic Regression (DLR), that explains how neurons in the brain can learn to predict sensory inputs using synaptic plasticity in their apical dendrites. DLR bridges the gap between theoretical models of Bayesian inference and experimental observations of neural plasticity, suggesting that apical dendrites enable neurons to learn complex probabilistic models of the world.

What Problem Does This Paper Address?

Neuroscience has long grappled with understanding how the brain integrates bottom-up sensory information with top-down predictions from higher brain areas. Specifically, the role of synapses on apical dendrites of pyramidal neurons in sensory cortices has been a mystery. While theoretical frameworks suggest these top-down inputs might represent probabilistic "priors" for sensory data, linking these abstract ideas to the actual biological mechanisms of synaptic plasticity has been challenging. This paper aims to bridge this gap by proposing a normative model for synaptic plasticity in apical dendrites that can learn these probabilistic predictions, drawing inspiration from experimental findings like the role of NMDA spikes.

Methods

The core of the paper is the development of a new learning rule called Dendritic Logistic Regression (DLR). This rule is derived from the goal of self-supervised learning of probabilistic predictions, aiming to approximate the performance of logistic regression, which is considered a gold standard for such tasks.

Key aspects of the method include:

  • Normative Model: DLR is framed as a "normative" model, meaning it's based on an assumed optimal functional goal: learning probabilistic predictions.
  • Apical Dendrite Plasticity: The model specifically focuses on synaptic plasticity in apical dendrites, incorporating experimental observations such as the importance of NMDA spikes.
  • Logistic Regression Approximation: DLR is designed to approximate logistic regression, a statistical method that maps weighted inputs to probabilities using a sigmoid function. The paper shows how synaptic plasticity rules can achieve this.
  • Stochastic Gradient Descent: The derivation of DLR involves minimizing a cross-entropy loss function using stochastic gradient descent, a common optimization technique.
  • Extension to Multiple Dendrites: The framework is extended to account for neurons with multiple apical dendrites, considering different interaction models between these branches (saturating and soft-XOR), reflecting experimental findings.

Datasets

The paper uses simulated datasets for training and evaluation. These datasets are generated to mimic scenarios where the target (e.g., a somatic spike) is probabilistically related to the input.

  • 2D Classification Task (Fig. 2): Data points are generated from two overlapping Gaussian distributions, representing a scenario where the input is not linearly separable, and the goal is to predict which distribution generated the data.
  • Moving Object Prediction Task (Fig. 3): This task involves predicting whether a moving object will cover a neuron's receptive field at a future time, based on noisy estimates of the object's initial position and velocity.
  • Complex Distributions Task (Fig. 4): This involves generating data from mixtures of Gaussians to create more complex probability landscapes, testing the ability of multi-branch dendrites to learn these.

No new benchmark datasets are introduced; rather, the paper uses synthetic data to illustrate the capabilities of its proposed model.

Key Insights and Ideas

  • Apical Dendrites as Probabilistic Predictors: The paper provides a compelling argument that apical dendrites, through specific plasticity mechanisms, are ideally suited for learning probabilistic predictions about sensory inputs.
  • NMDA Spikes as a Key Mechanism: The role of NMDA spikes in apical dendrites is highlighted as crucial for enabling this probabilistic learning, aligning with experimental observations.
  • DLR as a Biologically Plausible Gold Standard: DLR offers a way to bridge the gap between theoretical optimal learning (logistic regression) and the biological constraints of neural plasticity, suggesting a functional role for dendritic computation.
  • Dendritic Complexity Enables Complex Models: The paper demonstrates that the intricate structure of dendritic arborization, particularly with multiple branches, allows neurons to learn significantly more complex probability landscapes than single-branch models. This suggests a substantial functional advantage for dendritic computation.

Key Results

  • DLR Approximates Logistic Regression: Simulations show that DLR can achieve prediction performance very similar to theoretically optimal logistic regression on various tasks, including a challenging 2D classification problem. For instance, on the 2D classification task, DLR achieved a mean difference of 0.045 compared to logistic regression.
  • Faster Convergence for DLR: In classification tasks, DLR converges to near-optimal classifications much faster than the perceptron learning rule, especially when data is not linearly separable.
  • Predicting Future Sensory Input: The model successfully learns to predict the probability of a neuron's receptive field being covered by a moving object, closely matching the actual probabilities derived from full knowledge of the input distribution.
  • Multi-Branch Dendrites Learn Complex Landscapes: The paper shows that neurons with multiple apical dendrites, utilizing different interaction models (saturating, soft-XOR), can learn complex probability landscapes that a single dendrite cannot. For example, the soft-XOR interaction was particularly effective for learning alternating HIGH and LOW probability regions.
  • Increased Branches Improve Performance: Increasing the number of dendritic branches (from 2 to 4 or 6) with saturating interactions further improved the ability to model complex probability distributions across different tasks.

Limitations and Future Work

  • Simplifications in Models: The paper acknowledges that the models used for dendritic interactions (saturating, soft-XOR) are simplifications of complex biological processes.
  • Unknown Molecular Mechanisms: While DLR is consistent with experimental data, the precise molecular mechanisms underlying this plasticity are still not fully understood.
  • Diversity of Pyramidal Cells: The authors note that the vast diversity of pyramidal cell subtypes and NMDA receptors suggests that a single plasticity rule like DLR might not explain all observed phenomena.
  • Future Work: The authors suggest that comparing DLR's predictions with experimental data on specific apical dendrites can provide insights into whether their functional goal is indeed probability learning. They also point out that applying DLR to neurons with higher-dimensional inputs could reveal even more complex internal probabilistic models. The potential for dendritic arborization to enable neurons to learn substantially more complex internal models than networks of point neurons is highlighted as a significant area for future exploration.

Summary updated

Abstract

Abstract Sensory information is processed by the brain not in a simple feedforward fashion. Rather, bottom-up inputs are combined in pyramidal cells of sensory cortices with top-down information from higher brain areas that arrives through synapses in apical dendrites. The exact functional role of these top-down inputs has remained unknown. A promising abstract model posits that they provide probabilistic priors for bottom-up sensory inputs. We show that this hypothesis is consistent with a large number of experimental about synaptic plasticity in apical dendrites, in particular with the prominent role of NMDA-spikes. We identify conditions under which this synaptic plasticity could approximate the gold standard for self-supervised learning of probabilistic priors: logistic regression. Furthermore, this perspective suggests an additional functional role for the complex structure of the dendritic arborization plays: It enables the neuron to learn substantially more complex landscapes of probabilistic priors.
BibTeX
@misc{rao2021self,
  author = {A. Ravishankar Rao and Robert Legenstein and Anand Subramoney and Wolfgang Maass},
  title = {{Self-supervised learning of probabilistic prediction through synaptic plasticity in apical dendrites: A normative model}},
  year = {2021},
  doi = {10.1101/2021.03.04.433822},
  howpublished = {bioRxiv},
}
Figure