Learning poly-synaptic paths with traveling waves

Yoshiki Ito, Taro ToyoizumiPLOS Computational Biology 2021 · DOI

TL;DR

This paper proposes that traveling waves in the brain, combined with reward-based synaptic plasticity, can efficiently train complex, multi-step neural pathways. By synchronizing neural activity across distant neurons, these waves overcome limitations of local learning rules, enabling the brain to learn intricate functions like XOR and find optimal signal routes.

What Problem Does This Paper Address?

The human brain's immense complexity, with billions of neurons and trillions of connections, relies on efficient information processing. This processing often involves multi-step pathways (poly-synaptic paths) where signals travel through a sequence of neurons. A major challenge in understanding how these pathways are learned is the "credit assignment problem": how do we know which synapses in a long chain are responsible for a successful outcome, especially when some neurons in the chain might be inactive for extended periods? Standard synaptic plasticity rules, like Spike-Timing-Dependent Plasticity (STDP), are local and require co-activation of pre- and post-synaptic neurons. This makes it difficult to learn distant connections where intermediate neurons might not be firing. Previous work has suggested traveling waves play a role in learning, but the precise mechanism remained unclear. This paper aims to elucidate how traveling waves, in conjunction with reward-based learning, can facilitate the efficient learning of these complex, poly-synaptic pathways.

Methods

The authors employ a computational approach using spiking neural network simulations. They model individual neurons using the leaky integrate-and-fire model. A key innovation is the integration of a modified reward-modulated Spike-Timing-Dependent Plasticity (STDP) rule. This rule incorporates both a tonic dopaminergic signal (for reward-independent plasticity, preparing pathways) and a phasic dopaminergic signal (for reward-dependent plasticity, selecting successful pathways).

Crucially, they introduce a mechanism for traveling waves. These waves are modeled as fluctuations in the noise level of neurons, propagating across the network. This propagation is driven by a local field potential that spreads from stimulated neurons. The model allows for the comparison of networks with and without these traveling waves, as well as the isolation of the effects of the tonic dopaminergic signal.

They test their model in three distinct tasks:

  1. Reinforcing poly-synaptic paths: Learning to route a signal from a source to a target through a network with multiple potential paths.
  2. Finding a shortcut: Learning to favor direct, short paths over longer, indirect "detour" paths.
  3. Learning a nonlinear function (XOR): A classic computational challenge that requires non-trivial network architecture.

Datasets

Not discussed in the paper. The study uses simulated neural networks and does not rely on experimental datasets for training or evaluation. The tasks themselves serve as benchmarks for evaluating the model's performance.

Key Insights and Ideas

  • Traveling waves as a "path preparer": Traveling waves act as a mechanism to spread neural activity across the network, even to neurons that are not directly connected or are initially inactive. This "prepares" potential pathways by inducing coherent activity.
  • Synergy of tonic dopamine and waves: The tonic dopaminergic signal, acting as a reward-independent plasticity modulator, works synergistically with traveling waves. Together, they establish a broad repertoire of potential poly-synaptic paths.
  • Phasic dopamine as a "path selector": Once paths are prepared by waves and tonic dopamine, the phasic dopaminergic signal (triggered by actual rewards) can then selectively reinforce the most successful paths.
  • Overcoming local learning limitations: The combination of waves and reward-based plasticity effectively overcomes the limitations of purely local STDP, allowing for the learning of long-range, indirect connections.
  • Efficiency in finding optimal routes: Traveling waves help the network discover and strengthen the shortest or most efficient paths for signal transmission, a critical aspect for fast and reliable computation.
  • Enabling complex computations: The model demonstrates that this approach can learn non-linear functions like XOR, suggesting a biological mechanism for how complex cognitive functions might emerge.

Key Results

  • Task 1 (Poly-synaptic paths): The full model (waves + tonic dopamine) achieved the highest success rate in selectively reinforcing the correct path, significantly outperforming conventional models without waves or tonic dopamine. The conventional model failed to learn this task.
  • Task 2 (Shortcut finding): The wave conditions (with or without tonic dopamine) significantly reduced the latency of signal transmission to the target neuron compared to no-wave conditions. The wave + tonic dopamine condition showed the best performance, learning shortcut paths much faster and more reliably.
  • Task 3 (XOR function): Wave conditions demonstrated a significantly higher success rate in learning the XOR function compared to no-wave conditions. The paper highlights that the weak initial connections in this task necessitate the support of traveling waves for effective learning.

Limitations and Future Work

  • Simplified wave model: The authors acknowledge that their model of wave propagation is simplified, treating it as a local field effect rather than directly tied to membrane potential dynamics. Future work could explore more biologically realistic models of wave generation and propagation.
  • Global inhibition: The model uses global inhibition, whereas real brains have more complex, structured inhibitory circuits. Investigating the role of different inhibitory neuron types in wave dynamics and learning could be a future direction.
  • Separation of dynamics: The model separates neural activity dynamics from wave propagation. Future research could explore how these dynamics are more intricately linked in biological systems.

Despite these limitations, the study provides a compelling computational framework for understanding how traveling waves and dopaminergic signals might cooperate to enable efficient learning of complex neural circuits, offering insights into memory consolidation and adaptive behavior.

Summary updated

BibTeX
@article{ito2021learning,
  author = {Yoshiki Ito and Taro Toyoizumi},
  title = {{Learning poly-synaptic paths with traveling waves}},
  journal = {PLOS Computational Biology},
  year = {2021},
  doi = {10.1371/journal.pcbi.1008700},
}
Figure