Reverse-engineering recurrent neural network solutions to a hierarchical inference task for mice

Rylan Schaeffer, Mikail Khona, Leenoy Meshulam, International Brain Laboratory, Ila Fiete

Introduction: The Black Box of Hierarchical Inference

In both neuroscience and artificial intelligence, neural networks are often treated as "black boxes." While it is straightforward to train a Recurrent Neural Network (RNN) to perform a complex task, extracting the underlying mechanisms—its internal logic and dynamics—remains a significant challenge.

This paper tackles this challenge by focusing on hierarchical inference. Imagine a mouse in a laboratory setting tasked with deciding whether a faint visual stimulus (a light) appeared on its left or right. To complicate matters, the underlying rules change: during certain "blocks" of trials, the light is 80% likely to appear on the left, while in other blocks, it is 80% likely to appear on the right.

Generative model of the hierarchical inference task

To succeed, the subject (or the RNN) must simultaneously solve two problems across different timescales:

  1. Fast timescale (Stimulus inference): Where is the light located right now?
  2. Slow timescale (Context inference): Which probability "block" am I currently in?

The authors sought to understand how an RNN represents these two distinct variables simultaneously and whether a simple, human-readable "circuit" could be extracted to explain its behavior.

Methods: Distilling Dynamics

The researchers trained RNNs with 50 hidden units on a simulated version of the International Brain Laboratory (IBL) task. The input data consisted of noisy, Gaussian-sampled observations of stimulus side and contrast, alongside a feedback signal (reward or punishment) from the previous trial.

To evaluate the networks, the authors compared the RNNs' performance against Ideal Bayesian Baselines—mathematical models executing the statistically "perfect" calculation for the task.

To "crack" the RNN's internal code, they employed fixed-point analysis and Principal Component Analysis (PCA) to visualize the flow of neural activity. More importantly, they introduced a novel technique called Representations and Dynamics Distillation (RADD). Unlike standard knowledge distillation, which trains a smaller "student" model to mimic a larger "teacher's" outputs, RADD trains the tiny model to mimic the teacher's internal hidden state dynamics.

Key Insights: The Geometry of Belief

By analyzing the trained networks, the authors uncovered several fascinating mechanisms:

  • The 2D Geometry of Belief: Despite possessing 50 neurons, the RNN's decision-making process lives almost entirely on a low-dimensional 2D plane. One axis represents the network's belief about the current stimulus, while the other represents its belief about the current block.
  • Intentional Coupling: The axes representing stimulus and block beliefs are not perfectly perpendicular (orthogonal). This "tilt" is highly functional: it allows the network to automatically update its long-term block belief every time it processes a new stimulus.
  • Attractors as Memory: The network utilizes a line attractor to "remember" the block side across multiple trials. Within a single trial, it uses bistable attractors (two stable points) to commit to a binary choice (Left vs. Right).
  • RADD vs. Traditional Distillation: The study demonstrates that traditional output-matching distillation fails to capture the underlying dynamical phase portrait of the model. RADD successfully recovers a 2-unit model that is mathematically equivalent to the complex 50-unit version.

Key Results: Near-Optimal Performance

The trained RNNs achieved near-optimal performance, closely matching both the accuracy and the "chronometric curves" (decision time based on difficulty) of the ideal Bayesian observer.

RNN performance matching the ideal Bayesian observer

Using RADD, the authors successfully reduced the 50-unit RNN to a highly interpretable 2-unit RNN that maintained an 86.87% accuracy rate, nearly identical to the original network. This 2-unit model revealed specific "synaptic weights" corresponding to biological concepts, such as integration time constants. For instance, the unit tracking stimulus belief updates approximately 24 times faster than the unit tracking block belief, perfectly mirroring the hierarchical timescales of the task.

Limitations and Future Directions

While the IBL task requires complex hierarchical inference, it remains a controlled laboratory environment. Future work will need to explore how well RADD scales to higher-dimensional tasks, such as natural language processing or complex robotics.

Furthermore, the paper makes specific, testable predictions about how biological brains might represent this task—specifically regarding non-orthogonal representations. The next logical step is to validate these predictions against real calcium imaging or electrophysiology data from mice performing the IBL task. Finally, the authors propose that RADD could become a powerful tool for interpreting agents trained via reinforcement learning, helping to illuminate the opaque "why" behind an AI agent's learned policy.

Summary updated

Abstract

We study how recurrent neural networks (RNNs) solve a hierarchical inference task involving two latent variables and disparate timescales separated by 1-2 orders of magnitude. The task is of interest to the International Brain Laboratory, a global collaboration of experimental and theoretical neuroscientists studying how the mammalian brain generates behavior. We make four discoveries. First, RNNs learn behavior that is quantitatively similar to ideal Bayesian baselines. Second, RNNs perform inference by learning a two-dimensional subspace defining beliefs about the latent variables. Third, the geometry of RNN dynamics reflects an induced coupling between the two separate inference processes necessary to solve the task. Fourth, we perform model compression through a novel form of knowledge distillation on hidden representations -- Representations and Dynamics Distillation (RADD)-- to reduce the RNN dynamics to a low-dimensional, highly interpretable model. This technique promises a useful tool for interpretability of high dimensional nonlinear dynamical systems. Altogether, this work yields predictions to guide exploration and analysis of mouse neural data and circuity.
BibTeX
@inproceedings{schaeffer2020reverse,
  author = {Rylan Schaeffer and Mikail Khona and Leenoy Meshulam and International Brain Laboratory and Ila Fiete},
  title = {{Reverse-engineering recurrent neural network solutions to a hierarchical inference task for mice}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2020},
}
Figure