Evolutionary learning in the brain by heterosynaptic plasticity

Zedong Bi, Guozhang Chen, Dongping Yang, Yu Zhou, Liang TianbioRxiv 2021 · DOI

TL;DR

This paper proposes a novel model of brain learning that leverages evolutionary algorithms (EA) driven by heterosynaptic plasticity, a non-local form of synaptic modification. This biologically inspired EA framework successfully trains neural networks on complex tasks, offering a new perspective on how the brain might learn and adapt.

What Problem Does This Paper Address?

The fundamental question of how the brain learns and adapts its vast network of synapses to improve performance remains a major mystery in neuroscience. Current dominant models, like backpropagation, rely on "local plasticity rules" where synaptic changes depend only on the pre- and post-synaptic neuron's activity. However, experimental evidence for these purely local rules is not always conclusive, suggesting that other biological mechanisms might be at play. This paper tackles this gap by exploring two under-examined aspects: (1) alternative algorithmic paradigms beyond gradient descent, specifically evolutionary algorithms (EA), and (2) synaptic plasticity mechanisms beyond local rules, focusing on heterosynaptic plasticity.

Methods

The core of the paper is the development of an Evolutionary Algorithm (EA) model of brain learning that is biologically realized through heterosynaptic plasticity.

Here's a breakdown of the key components:

  • EA Framework: The model treats different "information routes" in the brain as agents in an EA. These agents compete, and those with higher "fitness" (task performance) are selected to reproduce. This is analogous to how biological evolution selects for advantageous traits.
  • Heterosynaptic Plasticity as the Learning Rule: Instead of traditional gradient-based updates, synaptic changes are driven by heterosynaptic plasticity. This involves non-local signaling pathways (like nitric oxide and calcium waves) that allow a synapse's modification to be influenced by the activity of other, unconnected synapses or neurons.
  • Dopamine as a Fitness Signal: Dopamine plays a crucial role in signaling the reward or performance of an agent, guiding the selection and reproduction process within the EA.
  • Biological Mechanisms: The model incorporates several biologically plausible mechanisms, including:
  • Dendrite Gating: Different dendritic branches of a neuron can be selectively activated, effectively gating information flow and allowing for distinct "agents" to be represented.
  • Meta-plasticity: This mechanism helps stabilize synapses, contributing to the "elitism" aspect of EA where the best performers are preserved.
  • Cooperative Plasticity: Synchronous inputs to neighboring synapses are modeled to enhance learning, reflecting observed phenomena in biological neural circuits.
  • Model-Free Approach: A key advantage highlighted is that the EA is "model-free," meaning it doesn't require explicit knowledge of the network architecture or task objective to learn.

Datasets

The paper evaluates its EA model on several standard benchmarks:

  • MNIST Classification: A common dataset for image classification.
  • Atari Games: A suite of classic video games used to test reinforcement learning capabilities.
  • Context-dependent decision-making task: This task, inspired by experimental data, involves making choices based on motion or color cues, with a context signal influencing which cue is relevant.

Key Insights and Ideas

  • Heterosynaptic Plasticity as a Powerful Learning Paradigm: The study demonstrates that heterosynaptic plasticity, often considered a secondary mechanism, can alone support a robust learning framework akin to evolutionary algorithms. This challenges the long-held focus on local plasticity rules.
  • Bridging AI and Neuroscience: The work elegantly connects machine learning paradigms (EA) with biological neural mechanisms, offering a new way to interpret brain functions like dopamine's role, memory replay, and dendritic spine plasticity.
  • Biological Plausibility of EA in the Brain: The model provides a concrete, biologically grounded mechanism for how an EA could operate within the brain, moving beyond abstract computational concepts.
  • Model-Free Learning for Generalization: The EA's model-free nature allows it to train diverse neural network architectures (spiking, analog, recurrent, feedforward) and tasks, highlighting its broad applicability and generalizability.
  • Binary Weights Achieve Comparable Performance: A significant finding is that EA can train networks with biologically plausible binary weights to achieve performance comparable to continuous-weight networks trained with state-of-the-art gradient-based methods, particularly in challenging tasks like Atari games.

Key Results

  • Broad Competence: The EA model successfully trained both spiking and analog neural networks with recurrent or feedforward architectures.
  • Powerful Capability: Deep neural networks trained with biologically plausible binary weights achieved performance comparable to continuous-weight networks on MNIST classification and Atari game playing tasks. For instance, in Atari games, EA-trained binary-weight networks outperformed or matched DQN and A3C in several games (e.g., "amidar," "assault," "asterix," "asteroids," "gravitar," "kangaroo," "seaquest," "venture," "zaxxon").
  • Mimicking Brain Dynamics: EA-trained networks exhibited dynamics analogous to those observed in the brain during cognitive tasks, supporting the biological plausibility of the proposed mechanism.
  • Off-line Replay Compatibility: The EA framework is compatible with off-line replay, suggesting a role in consolidating memories during rest or sleep.

Limitations and Future Work

  • Learning Speed: The authors acknowledge that EA's learning speed can be slower than gradient-based methods, especially for tasks requiring many epochs (e.g., MNIST classification). They suggest that combining EA with homosynaptic mechanisms or employing curriculum learning could accelerate this.
  • Cross-over Operation: The current model primarily uses mutation. The authors note that including cross-over operations, a standard component of EA, could further enhance learning speed.
  • Experimental Testing: While the model is biologically inspired, direct experimental validation of some proposed mechanisms (e.g., stabilization of small synapses) remains to be done.
  • Optogenetic Therapy for AD: The work suggests potential implications for Alzheimer's disease, proposing that optogenetic stimulation of SST interneurons could ameliorate deficits related to rigid information routes. This opens avenues for therapeutic research.

Summary updated

Abstract

Summary How the brain modifies synapses to improve the performance of complicated networks remains one of the biggest mysteries in neuroscience. Canonical models suppose synaptic weights change according to pre- and post-synaptic activities (i.e., local plasticity rules), implementing gradient-descent algorithms. However, the lack of experimental evidence to confirm these models suggests that there may be important ingredients neglected by these models. For example, heterosynaptic plasticity, non-local rules mediated by inter-cellular signaling pathways, and the biological implementation of evolutionary algorithms (EA), another machine-learning paradigm that successfully trains large-scale neural networks, are seldom explored. Here we propose and systematically investigate an EA model of brain learning with non-local rules alone. Specifically, a population of agents are represented by different information routes in the brain, whose task performances are evaluated through gating on individual routes alternatively. The selection and reproduction of agents are realized by dopamine-guided heterosynaptic plasticity. Our EA model provides a framework to re-interpret the biological functions of dopamine, meta-plasticity of dendritic spines, memory replay, and the cooperative plasticity between the synapses within a dendritic neighborhood from a new and coherent aspect. Neural networks trained with the model exhibit analogous dynamics to the brain in cognitive tasks. Our EA model manifests broad competence to train spiking or analog neural networks with recurrent or feedforward architecture. Our EA model also demonstrates its powerful capability to train deep networks with biologically plausible binary weights in MNIST classification and Atari-game playing tasks with performance comparable with continuous-weight networks trained by gradient-based methods. Overall, our work leads to a fresh understanding of the brain learning mechanism unexplored by local rules and gradient-based algorithms.
BibTeX
@misc{bi2021evolutionary,
  author = {Zedong Bi and Guozhang Chen and Dongping Yang and Yu Zhou and Liang Tian},
  title = {{Evolutionary learning in the brain by heterosynaptic plasticity}},
  year = {2021},
  doi = {10.1101/2021.12.14.472260},
  howpublished = {bioRxiv},
}
Figure