Targeted Neural Dynamical Modeling

Cole Lincoln Hurwitz, Akash Srivastava, Kai Xu, Justin Jude, Matt Perich, Lee E. Miller, Matthias H. Hennig

Understanding how the brain's electrical activity translates into physical movement is a central challenge in neuroscience. Researchers often record the activity of hundreds of neurons simultaneously while an animal performs a task, hoping to find the "latent dynamics"—the underlying, low-dimensional patterns that govern this high-dimensional neural "noise."

Recently presented at NeurIPS 2021, Targeted Neural Dynamical Modeling (TNDM) introduces a novel framework that combines the flexibility of deep learning with the structure of state-space models. TNDM provides a way to "sift" neural signals, effectively separating the dynamics that drive behavior from those that do not.

The Challenge with Existing Models

Current methods for modeling neural population activity typically face a frustrating trade-off:

  1. Unsupervised models (e.g., LFADS): These models are excellent at reconstructing overall neural activity but tend to "mix" different signals together. It is difficult to isolate which part of the brain's activity is actively moving a limb versus which part is related to internal, task-agnostic states like hunger or arousal.
  2. Supervised/Linear models (e.g., PSID): These models focus specifically on behavior but are limited by strict linear assumptions. They often struggle with complex, nonlinear movements and unrealistically assume that neural activity and behavior occur at the exact same moment (i.e., no time lag).

TNDM bridges this gap by offering a model that is nonlinear, accommodates temporal lags, and explicitly disentangles behavior-driving signals from the rest of the neural activity.

The TNDM Architecture

TNDM is implemented as a Sequential Variational Autoencoder (VAE) and introduces several key architectural innovations to achieve its disentanglement:

TNDM Architecture and Graphical Model
  • Partitioned Latent Space: The model utilizes two separate Recurrent Neural Networks (RNNs) to generate two distinct sets of latent variables: relevant ($z_r$) and irrelevant ($z_i$) dynamics.
  • Causal Behavioral Decoder: Unlike previous models that force a rigid 1:1 mapping at each time step, TNDM employs a flexible linear decoder that allows for time lags. It enforces causality, ensuring that only past and present neural states can influence current behavior.
  • Disentanglement Penalty: To prevent the relevant and irrelevant latent spaces from blending into each other, the authors apply a cross-correlation penalty on the initial conditions of the RNNs. This forces the model to strictly use the separate channels for their intended purposes.
  • Joint Objective Function: TNDM is trained to simultaneously maximize the likelihood of the neural spikes (using a Poisson distribution) and minimize the error of the behavioral data reconstruction (using Mean Squared Error).

Key Insights and Results

The authors validated TNDM using simulated data (synthetic spike trains from a Lorenz system) and real electrophysiology recordings from the primary motor cortex (M1) and premotor cortex (PMd) of a macaque monkey performing a center-out reaching task.

The experiments revealed several critical insights:

  • Nonlinearity is Essential: Linear models like PSID failed to accurately capture the dynamics of multi-directional reaching. TNDM’s RNN-based approach modeled these complex trajectories far more effectively.
  • Behavioral Constraints Improve Neural Fit: A common assumption is that forcing a model to explain behavior will degrade its ability to explain raw neural data. TNDM disproves this; its neural reconstruction was just as good as, and sometimes better than, purely unsupervised models like LFADS.
  • Interpretability via Separation: Visualizing the latent space showed that TNDM’s "relevant" factors clearly clustered by reach direction, while the "irrelevant" factors captured task-agnostic dynamics, such as the general burst of activity at the onset of movement.
Performance comparison of TNDM, LFADS, and PSID

As shown in the performance metrics, TNDM achieves superior predictive power efficiently. On the M1 dataset, TNDM reached a behavioral $R^2$ of ~0.90 using only two relevant latent factors, vastly outperforming the linear PSID model. Furthermore, TNDM achieved a lower Root Mean Square Error (RMSE) for firing rate prediction compared to LFADS, demonstrating that behavioral supervision helps the model discover more accurate underlying neural dynamics.

Limitations and Future Directions

While TNDM represents a significant step forward, the authors note a few limitations. The model exhibits some hyperparameter sensitivity; certain settings can lead to biologically implausible oscillations, requiring careful tuning. Additionally, the causal linear decoder is parameter-inefficient, scaling quadratically with trial length, which could bottleneck very long recordings.

Currently, TNDM focuses on continuous movement (like hand position). Future work will look to extend this framework to discrete behaviors, such as decision-making, or higher-dimensional outputs like muscle activity (EMG).

Summary updated

Abstract

Latent dynamics models have emerged as powerful tools for modeling and interpreting neural population activity. Recently, there has been a focus on incorporating simultaneously measured behaviour into these models to further disentangle sources of neural variability in their latent space. These approaches, however, are limited in their ability to capture the underlying neural dynamics (e.g. linear) and in their ability to relate the learned dynamics back to the observed behaviour (e.g. no time lag). To this end, we introduce Targeted Neural Dynamical Modeling (TNDM), a nonlinear state-space model that jointly models the neural activity and external behavioural variables. TNDM decomposes neural dynamics into behaviourally relevant and behaviourally irrelevant dynamics; the relevant dynamics are used to reconstruct the behaviour through a flexible linear decoder and both sets of dynamics are used to reconstruct the neural activity through a linear decoder with no time lag. We implement TNDM as a sequential variational autoencoder and validate it on simulated recordings and recordings taken from the premotor and motor cortex of a monkey performing a center-out reaching task. We show that TNDM is able to learn low-dimensional latent dynamics that are highly predictive of behaviour without sacrificing its fit to the neural data.
BibTeX
@inproceedings{hurwitz2021targeted,
  author = {Cole Lincoln Hurwitz and Akash Srivastava and Kai Xu and Justin Jude and Matt Perich and Lee E. Miller and Matthias H. Hennig},
  title = {{Targeted Neural Dynamical Modeling}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=dnDkuSzNh8},
}
Figure