Understanding how the brain's electrical activity translates into physical movement is a central challenge in neuroscience. Researchers often record the activity of hundreds of neurons simultaneously while an animal performs a task, hoping to find the "latent dynamics"—the underlying, low-dimensional patterns that govern this high-dimensional neural "noise."
Recently presented at NeurIPS 2021, Targeted Neural Dynamical Modeling (TNDM) introduces a novel framework that combines the flexibility of deep learning with the structure of state-space models. TNDM provides a way to "sift" neural signals, effectively separating the dynamics that drive behavior from those that do not.
The Challenge with Existing Models
Current methods for modeling neural population activity typically face a frustrating trade-off:
- Unsupervised models (e.g., LFADS): These models are excellent at reconstructing overall neural activity but tend to "mix" different signals together. It is difficult to isolate which part of the brain's activity is actively moving a limb versus which part is related to internal, task-agnostic states like hunger or arousal.
- Supervised/Linear models (e.g., PSID): These models focus specifically on behavior but are limited by strict linear assumptions. They often struggle with complex, nonlinear movements and unrealistically assume that neural activity and behavior occur at the exact same moment (i.e., no time lag).
TNDM bridges this gap by offering a model that is nonlinear, accommodates temporal lags, and explicitly disentangles behavior-driving signals from the rest of the neural activity.
The TNDM Architecture
TNDM is implemented as a Sequential Variational Autoencoder (VAE) and introduces several key architectural innovations to achieve its disentanglement:
- Partitioned Latent Space: The model utilizes two separate Recurrent Neural Networks (RNNs) to generate two distinct sets of latent variables: relevant ($z_r$) and irrelevant ($z_i$) dynamics.
- Causal Behavioral Decoder: Unlike previous models that force a rigid 1:1 mapping at each time step, TNDM employs a flexible linear decoder that allows for time lags. It enforces causality, ensuring that only past and present neural states can influence current behavior.
- Disentanglement Penalty: To prevent the relevant and irrelevant latent spaces from blending into each other, the authors apply a cross-correlation penalty on the initial conditions of the RNNs. This forces the model to strictly use the separate channels for their intended purposes.
- Joint Objective Function: TNDM is trained to simultaneously maximize the likelihood of the neural spikes (using a Poisson distribution) and minimize the error of the behavioral data reconstruction (using Mean Squared Error).
Key Insights and Results
The authors validated TNDM using simulated data (synthetic spike trains from a Lorenz system) and real electrophysiology recordings from the primary motor cortex (M1) and premotor cortex (PMd) of a macaque monkey performing a center-out reaching task.
The experiments revealed several critical insights:
- Nonlinearity is Essential: Linear models like PSID failed to accurately capture the dynamics of multi-directional reaching. TNDM’s RNN-based approach modeled these complex trajectories far more effectively.
- Behavioral Constraints Improve Neural Fit: A common assumption is that forcing a model to explain behavior will degrade its ability to explain raw neural data. TNDM disproves this; its neural reconstruction was just as good as, and sometimes better than, purely unsupervised models like LFADS.
- Interpretability via Separation: Visualizing the latent space showed that TNDM’s "relevant" factors clearly clustered by reach direction, while the "irrelevant" factors captured task-agnostic dynamics, such as the general burst of activity at the onset of movement.
As shown in the performance metrics, TNDM achieves superior predictive power efficiently. On the M1 dataset, TNDM reached a behavioral $R^2$ of ~0.90 using only two relevant latent factors, vastly outperforming the linear PSID model. Furthermore, TNDM achieved a lower Root Mean Square Error (RMSE) for firing rate prediction compared to LFADS, demonstrating that behavioral supervision helps the model discover more accurate underlying neural dynamics.
Limitations and Future Directions
While TNDM represents a significant step forward, the authors note a few limitations. The model exhibits some hyperparameter sensitivity; certain settings can lead to biologically implausible oscillations, requiring careful tuning. Additionally, the causal linear decoder is parameter-inefficient, scaling quadratically with trial length, which could bottleneck very long recordings.
Currently, TNDM focuses on continuous movement (like hand position). Future work will look to extend this framework to discrete behaviors, such as decision-making, or higher-dimensional outputs like muscle activity (EMG).
Summary updated