Charting and navigating the space of solutions for recurrent neural\n networks

Elia Turner, Kabir Dabholkar, Omri Barak

Abstract: Recurrent Neural Networks (RNNs) were recently successfully used to model the way neural activity drives task-related behavior in animals, operating under the implicit assumption that the obtained solutions are universal. Observations in both neuroscience and machine learning challenge this assumption. Animals can approach a given task with a variety of strategies, and training machine learning algorithms introduces the phenomenon of underspecification. These observations imply that every task is associated with a space of solutions. To date, the structure of this space is not understood, limiting the approach of comparing RNNs with neural data. Here, we characterize the space of solutions associated with various tasks. We first study a simple two-neuron network on a task that leads to multiple solutions. We trace the nature of the final solution back to the network's initial connectivity and identify discrete dynamical regimes that underlie this diversity. We then examine three neuroscience-inspired tasks: Delayed and interval discrimination, and Time reproduction. For each task, we find a rich set of solutions. Variability can be found directly in the neural activity of the networks, and additionally by testing the trained networks' ability to extrapolate, as a perturbation to a system often reveals hidden structure. Furthermore, we relate extrapolation patterns to specific dynamical objects and effective algorithms found by the networks. We introduce a tool to derive the reduced dynamics of networks by generating a compact directed graph describing the essence of the dynamics with regards to behavioral inputs and outputs. Using this representation, we can partition the solutions to each task into a handful of types and partially predict them from neural features. Our results shed light on the concept of the space of solutions and its uses in Machine learning and in Neuroscience.

Introduction: The Problem of Underspecification

In both machine learning and neuroscience, there is often an implicit assumption of universality: the idea that if a model (or an animal) solves a specific task, it must be using a specific, "optimal" mechanism. However, modern ML models are "over-parameterized," meaning there are countless ways to set the weights to get the same output—a phenomenon called underspecification.

This creates a major challenge for computational neuroscientists. If an RNN mimics a biological subject's behavior, how do we know it has captured the correct underlying algorithm? If the RNN could have solved the task in ten different ways, comparing just one of them to biological data is scientifically risky. This paper seeks to "chart" this space of solutions to understand the variety of algorithms that exist for a given task and what biases a network toward one or another.

Methods: Probing Hidden Structures

The authors employ a dynamical systems lens to look under the hood of RNNs. Their approach involves three main stages:

  1. Extrapolation Testing: They don't just test networks on the data they were trained on. They "push" the networks by giving them inputs outside the training range (e.g., much longer delays). This often breaks the "illusion" of identical behavior and reveals the hidden dynamical structures (like fixed points vs. limit cycles) that the network is actually using.
  2. Reduced Dynamics Tool: The authors introduced a novel algorithm to simplify complex network activity. It compresses high-dimensional neural trajectories into a compact directed graph. In these graphs, nodes represent areas in phase space with specific outputs, and edges represent how the system transitions between them.
  3. Feature-Based Classification: They extracted "neural features" (like the curvature of a trajectory or the speed of neural activity) during the training phase and used a Random Forest classifier to see if these features could predict which "solution type" (graph) the network belonged to.

Key Insights: Discrete Islands in Parameter Space

By studying a simple 2-neuron network, the researchers demonstrated that solutions aren't always on a smooth sliding scale. Instead, networks often fall into "discrete islands" or regimes separated by bifurcations—mathematical tipping points where the nature of the solution fundamentally changes. As shown in the figure above, the initial connectivity of the network can determine whether it settles into a stable origin, a non-trivial fixed point, or a limit cycle.

Furthermore, while Vanilla RNNs, GRUs, and LSTMs can all solve the same task, they have different "inductive biases." For example, GRUs might prefer one specific type of timing solution that Vanilla RNNs almost never find. Despite this diversity, solutions are not random; a handful of solution types (represented by the reduced graphs) usually account for over 90% of all trained networks.

Key Results

  • Solution Partitioning: For the Interval Reproduction task, the authors identified four major solution types that accounted for 93% of all networks trained.
  • Neural Signatures: The researchers demonstrated that "experimentally accessible" neural features could predict the network's long-term asymptotic behavior (the reduced graph) with above-chance accuracy (Cohen’s kappa between 0.37 and 0.61).
  • Task Constraints: They confirmed that while some tasks allow for a "rich space" of solutions, others (like Context-Dependent Integration) are highly constrained and lead to a single, universal solution.

Limitations and Future Work

The study mostly used identical hyperparameters (like network size), and the authors acknowledge that changing the number of neurons or the learning rate might expand the solution space even further. Additionally, the "reduced dynamics" tool only maps parts of the phase space that the network actually visits. A promising future direction is using these "maps" to track how a network travels through the space of solutions during the training process itself.

Summary updated

Abstract

Recurrent Neural Networks (RNNs) were recently successfully used to model the\nway neural activity drives task-related behavior in animals, operating under\nthe implicit assumption that the obtained solutions are universal. Observations\nin both neuroscience and machine learning challenge this assumption. Animals\ncan approach a given task with a variety of strategies, and training machine\nlearning algorithms introduces the phenomenon of underspecification. These\nobservations imply that every task is associated with a space of solutions. To\ndate, the structure of this space is not understood, limiting the approach of\ncomparing RNNs with neural data. Here, we characterize the space of solutions\nassociated with various tasks. We first study a simple two-neuron network on a\ntask that leads to multiple solutions. We trace the nature of the final\nsolution back to the network's initial connectivity and identify discrete\ndynamical regimes that underlie this diversity. We then examine three\nneuroscience-inspired tasks: Delayed and interval discrimination, and Time\nreproduction. For each task, we find a rich set of solutions. Variability can\nbe found directly in the neural activity of the networks, and additionally by\ntesting the trained networks' ability to extrapolate, as a perturbation to a\nsystem often reveals hidden structure. Furthermore, we relate extrapolation\npatterns to specific dynamical objects and effective algorithms found by the\nnetworks. We introduce a tool to derive the reduced dynamics of networks by\ngenerating a compact directed graph describing the essence of the dynamics with\nregards to behavioral inputs and outputs. Using this representation, we can\npartition the solutions to each task into a handful of types and partially\npredict them from neural features. Our results shed light on the concept of the\nspace of solutions and its uses in Machine learning and in Neuroscience.\n
BibTeX
@inproceedings{turner2021charting,
  author = {Elia Turner and Kabir Dabholkar and Omri Barak},
  title = {{Charting and navigating the space of solutions for recurrent neural\textbackslash\{\}n networks}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=SQm_poGrlj},
}
Figure