Learning compositional sequences with multiple time scales through a hierarchical network of spiking neurons

Amadeus Maes, Mauricio Barahona, Claudia Clopath

TL;DR

This paper introduces a novel hierarchical spiking neural network model that can learn and replay complex sequences composed of smaller motifs. Unlike serial models, this hierarchical approach allows for independent learning of motifs and their ordering (syntax), leading to faster learning, more efficient relearning, increased capacity, and greater robustness to perturbations.

What Problem Does This Paper Address?

Many natural behaviors, from animal movements to human language, are compositional. This means they are built from smaller, recurring units (motifs) arranged in a specific order (syntax). This structure implies multiple timescales: a fast timescale for the motifs themselves and a slower timescale for their arrangement. While the brain is known to exhibit hierarchical organization, current computational models for learning sequential behavior have largely relied on serial approaches. These serial models struggle with flexibility, robustness, and efficient relearning, as changing the sequence often requires relearning everything from scratch. This paper addresses the challenge of building a biologically plausible neural network model that can effectively learn and represent these multi-timescale compositional sequences, mirroring the brain's apparent hierarchical processing.

Methods

The authors propose a hierarchical spiking neural network architecture comprising four main components:

  • Fast Clock Network: Responsible for the rapid, within-motif dynamics.
  • Slow Clock Network: Manages the slower, between-motif temporal organization (syntax).
  • Read-out Networks (Motif Networks): Each dedicated to learning and producing a specific motif.
  • Interneuron Networks (Syntax Networks): These coordinate the ordering of motifs based on the slow clock's input.

Crucially, the model uses bio-plausible, local plasticity rules, specifically a symmetric Spike-Timing-Dependent Plasticity (STDP) rule, for learning both the motifs (via synapses from the fast clock to read-out networks) and the syntax (via synapses from the slow clock to interneurons). This contrasts with methods requiring global optimization like backpropagation. The architecture allows for a dissociation of motif learning and syntax learning, with the fast clock controlling motif dynamics and the slow clock, via interneurons, controlling motif order.

Datasets

Not discussed in the paper. The study uses simulated target sequences (e.g., AAB, ABA, ABCD, EBCF) for training and evaluation, rather than real-world datasets.

Key Insights and Ideas

  • Decoupled Learning: The hierarchical structure naturally separates the learning of individual motifs from the learning of their temporal order (syntax). This is a key advantage over serial models where these are intertwined.
  • Efficiency in Relearning: The model can efficiently reorder existing motifs into new sequences without needing to relearn the motifs themselves, demonstrating flexibility.
  • Resource Optimization: Storing multiple sequences requires significantly fewer neurons and synapses in the hierarchical model compared to a serial model that needs to duplicate entire read-out networks.
  • Robustness to Perturbations: The separation of timescales in the hierarchical model provides inherent robustness. Perturbing the fast clock doesn't necessarily disrupt the slow clock's ability to maintain the overall sequence structure.
  • Variability Trade-off: The hierarchical model achieves high motif fidelity (consistent motif execution) at the cost of increased variability in the timing between motifs. This suggests a neural mechanism for prioritizing precise execution of core components over exact timing between them.

Key Results

  • Faster Learning: The hierarchical model learns target sequences approximately three times faster than the serial model.
  • Improved Motif Fidelity: The hierarchical model produces more consistent and accurate replays of individual motifs compared to the serial model, especially when motifs are repeated within a sequence.
  • Increased Capacity: For storing multiple sequences, the hierarchical model requires substantially fewer neurons and synapses. For example, storing two sequences required 7350 neurons and 4.13 million synapses, compared to 7500 neurons and 5.76 million synapses for the serial model.
  • Enhanced Robustness: When the fast clock is perturbed, the hierarchical model can still recover and replay later motifs, whereas a serial model's sequence breaks down completely.
  • Variable Timing: The model exhibits higher variability in the time intervals between motifs, while maintaining high similarity within motifs across repetitions.

Limitations and Future Work

  • Parameter Tuning: The model has free parameters (topology, plasticity) that require tuning to specific tasks, making direct quantitative links to experimental data challenging.
  • Speed Control: Varying the speed of sequence execution is not as straightforward in the hierarchical model as in the serial model. The authors suggest potential mechanisms for speed control, such as coordination between clocks or downstream networks.
  • Interneuron Network Specificity: The hardwired lateral connectivity in the interneuron network is a limitation, though it appears robust to parameter variations.
  • Future Directions: The authors suggest exploring how to infer the hierarchical structure from temporal data generated by the model using data analysis techniques. They also propose investigating how the model could be extended to handle more complex scenarios, such as learning sequences with variable motif durations and inter-motif intervals, which they briefly demonstrate.

Summary updated

Abstract

Author summary The brain has the ability to learn and execute sequential behaviour on multiple time scales. This behaviour is often compositional: a set of simple behaviours is concatenated to create a complex behaviour. Technological improvements increasingly shine light on the building blocks of compositional behaviour, yet the underlying neural mechanisms remain unclear. Here, we propose a hierarchical model to study the learning and execution of compositional sequences, using bio-plausible neurons and learning rules. We compare the hierarchical model with a serial version of the model. We demonstrate that the hierarchical model is more flexible, efficient and robust by exploiting the compositional nature of the sequences.
BibTeX
@article{maes2021learning,
  author = {Amadeus Maes and Mauricio Barahona and Claudia Clopath},
  title = {{Learning compositional sequences with multiple time scales through a hierarchical network of spiking neurons}},
  journal = {PLOS Computational Biology},
  year = {2021},
  doi = {10.1371/journal.pcbi.1008866},
}
Figure