Structured flexibility in recurrent neural networks via neuromodulation

Julia C. Costacurta, Shaunak Bhandarkar, David M. Zoltowski, Scott W. LindermanbioRxiv 2024 · DOI

TL;DR

This paper introduces the Neuromodulated RNN (NM-RNN), a novel recurrent neural network architecture that incorporates a biologically-inspired neuromodulatory system to dynamically adjust its internal weights. This "structured flexibility" allows NM-RNNs to outperform standard low-rank RNNs on tasks requiring adaptation and generalization, bridging the gap between biologically realistic models and current artificial neural networks.

What Problem Does This Paper Address?

Traditional recurrent neural networks (RNNs), while powerful, often rely on fixed weight matrices to represent synaptic connectivity. This is a simplification, as biological brains exhibit dynamic synaptic plasticity, heavily influenced by neuromodulatory signals. These signals can alter synaptic strengths, reconfiguring circuit dynamics and enabling flexible behavior. The authors aim to bridge this gap by introducing a computational model that captures this neuromodulatory influence, seeking to enhance RNNs' ability to adapt, generalize, and potentially capture more complex biological computations.

Methods

The core innovation is the Neuromodulated RNN (NM-RNN). It consists of two interconnected subnetworks:

  1. Output-generating subnetwork: A low-rank RNN (LR-RNN) responsible for producing the network's output. The "low-rank" aspect is crucial for interpretability and efficient computation.
  2. Neuromodulatory subnetwork: A smaller, full-rank RNN that generates a low-dimensional, time-varying neuromodulatory signal.

This neuromodulatory signal dynamically scales the individual rank-1 components of the output-generating RNN's recurrent weight matrix. This allows the network to flexibly adjust its internal dynamics based on the task context, mimicking how neuromodulators influence biological circuits. The authors also provide theoretical analysis, showing how this mechanism can endow networks with gating capabilities similar to those found in Long Short-Term Memory (LSTM) networks.

Datasets

The authors used synthetic datasets generated for the following tasks:

  • Measure-Wait-Go (MWG) task: A timing task where networks must reproduce a specific interval duration.
  • Multitask learning set: Including DelayPro, DelayAnti, MemoryPro, and MemoryAnti tasks, which involve processing sensory information and context to produce specific outputs.
  • Element Finder Task (EFT): A sequence-based task requiring the network to recall a specific element from a sequence based on a query index.

Key Insights and Ideas

  • Biologically-inspired flexibility: Neuromodulation, a key feature of biological brains, can be computationally harnessed to create more adaptable and flexible artificial neural networks.
  • Structured weight modulation: Instead of arbitrary weight changes, the NM-RNN uses a low-dimensional neuromodulatory signal to scale specific components of the low-rank weight matrix, providing a structured and interpretable form of dynamic control.
  • Gating mechanisms: The neuromodulatory gain scaling naturally leads to gating-like behaviors, drawing parallels to established artificial neural network architectures like LSTMs.
  • Bridging the gap: The NM-RNN offers a promising avenue for connecting highly detailed biophysical models with more abstract, task-optimized RNNs.

Key Results

  • Improved performance on timing tasks: NM-RNNs consistently achieved lower L2 loss than comparable LR-RNNs on the Measure-Wait-Go task, both for trained and extrapolated intervals, demonstrating better generalization.
  • Enhanced multitask learning: In a multitask setting, NM-RNNs showed superior performance when retraining on a new task compared to LR-RNNs, suggesting they can more effectively reconfigure existing dynamics.
  • LSTM-like capabilities: The NM-RNN demonstrated performance comparable to LSTMs on the Element Finder Task, a task known to be challenging for standard RNNs, highlighting its ability to capture long-term dependencies.
  • Interpretable dynamics: The authors visualized the neuromodulatory signals and internal states, showing how they dynamically adapt to task demands and contribute to flexible computation.

Limitations and Future Work

  • Scale of networks: The tested networks were relatively small (N ≈ 100 neurons). Future work could explore larger-scale networks.
  • Comparison to neural data: The study primarily focused on computational performance. Future work could compare NM-RNNs' dynamics to actual neural recordings to draw stronger biological conclusions.
  • Sparsity and value range: The authors suggest exploring sparser neuromodulatory effects and allowing neuromodulatory signals to take negative values, mirroring biological neuromodulators more closely.
  • Learning dynamics: The current study analyzes networks post-training. Investigating how neuromodulation impacts the learning process itself is a promising future direction.
  • Other signaling mechanisms: Exploring other biological signaling mechanisms beyond neuromodulators could lead to further model extensions.

Summary updated

Abstract

Abstract The goal of theoretical neuroscience is to develop models that help us better understand biological intelligence. Such models range broadly in complexity and biological detail. For example, task-optimized recurrent neural networks (RNNs) have generated hypotheses about how the brain may perform various computations, but these models typically assume a fixed weight matrix representing the synaptic connectivity between neurons. From decades of neuroscience research, we know that synaptic weights are constantly changing, controlled in part by chemicals such as neuromodulators. In this work we explore the computational implications of synaptic gain scaling, a form of neuromodulation, using task-optimized low-rank RNNs. In our neuromodulated RNN (NM-RNN) model, a neuromodulatory subnetwork outputs a low-dimensional neuromodulatory signal that dynamically scales the low-rank recurrent weights of an output-generating RNN. In empirical experiments, we find that the structured flexibility in the NM-RNN allows it to both train and generalize with a higher degree of accuracy than low-rank RNNs on a set of canonical tasks. Additionally, via theoretical analyses we show how neuromodulatory gain scaling endows networks with gating mechanisms commonly found in artificial RNNs. We end by analyzing the low-rank dynamics of trai ned NM-RNNs, to show how task computations are distributed.
BibTeX
@misc{costacurta2024structured,
  author = {Julia C. Costacurta and Shaunak Bhandarkar and David M. Zoltowski and Scott W. Linderman},
  title = {{Structured flexibility in recurrent neural networks via neuromodulation}},
  year = {2024},
  doi = {10.1101/2024.07.26.605315},
  howpublished = {bioRxiv},
}
Figure