Abstract
Latent dynamics models have emerged as powerful tools for modeling and interpreting neural population activity. Recently, there has been a focus on incorporating simultaneously measured behaviour into these models to further disentangle sources of neural variability in their latent space. These approaches, however, are limited in their ability to capture the underlying neural dynamics (e.g. linear) and in their ability to relate the learned dynamics back to the observed behaviour (e.g. no time lag). To this end, we introduce Targeted Neural Dynamical Modeling (TNDM), a nonlinear state-space model that jointly models the neural activity and external behavioural variables. TNDM decomposes neural dynamics into behaviourally relevant and behaviourally irrelevant dynamics; the relevant dynamics are used to reconstruct the behaviour through a flexible linear decoder and both sets of dynamics are used to reconstruct the neural activity through a linear decoder with no time lag. We implement TNDM as a sequential variational autoencoder and validate it on simulated recordings and recordings taken from the premotor and motor cortex of a monkey performing a center-out reaching task. We show that TNDM is able to learn low-dimensional latent dynamics that are highly predictive of behaviour without sacrificing its fit to the neural data.
Introduction
A central goal in systems neuroscience is to identify low-dimensional "latent dynamics" that explain the complex activity of large neural populations. While unsupervised models like Latent Factor Analysis via Dynamical Systems (LFADS) excel at capturing nonlinear dynamics, they struggle to distinguish between signals driving behavior and those related to internal states like arousal or thirst. Conversely, supervised models like Preferential Subspace Identification (PSID) explicitly link neural activity to behavior but are often constrained by linear assumptions and a rigid "no-lag" relationship between brain activity and movement.
Targeted Neural Dynamical Modeling (TNDM) addresses this gap by providing a nonlinear, interpretable framework that separates behaviorally relevant signals from background neural activity while accounting for biological time lags.
Methodology
TNDM is implemented as a sequential Variational Autoencoder (VAE) designed to partition the latent space into two distinct components: behaviorally relevant factors ($z_r$) and behaviorally irrelevant factors ($z_i$).
As shown in the architecture above, TNDM utilizes dual Recurrent Neural Network (RNN) generators to evolve these dynamics independently. A key innovation is the causal linear decoder for behavior, which allows the model to learn relationships where past neural activity influences current behavior—a more biologically plausible assumption than instantaneous mapping. To ensure effective separation, a cross-correlation penalty is applied to the initial conditions ($g_0$) of the two latent paths, preventing the mixing of relevant and irrelevant information.
The model was validated using a synthetic Lorenz system (providing ground-truth benchmarks) and electrophysiological recordings from the motor (M1) and premotor (PMd) cortex of a macaque monkey performing a center-out reaching task.
Results and Discussion
TNDM demonstrates that behavior can be explained by a significantly lower-dimensional subspace than previously suggested by unsupervised methods.
Superior Behavior Prediction and Neural Fit
On M1 data, TNDM achieved a behavioral reconstruction ($R^2$) of approximately 0.90 using only 3 latent factors. As illustrated in Figure 4(a), LFADS requires nearly 8 factors to reach comparable performance, while PSID remains significantly lower. Crucially, this focus on behavior does not degrade the model's ability to fit raw neural data; Figure 4(b) shows that TNDM maintains a Poisson log-likelihood nearly identical to the unsupervised gold standard, LFADS.
The Importance of Nonlinearity and Causality
The failure of linear models like PSID in complex reaching tasks underscores the necessity of TNDM's nonlinear RNN-based approach. Furthermore, by allowing for time lags, TNDM captures the natural lead-time of motor cortex signals relative to physical movement. Models that enforce a "no-lag" constraint often suffer in their ability to accurately reconstruct underlying neural activity.
Limitations
Despite its strengths, TNDM is sensitive to hyperparameter settings, which can occasionally produce biologically implausible oscillations. Additionally, the causal behavior decoder's complexity scales quadratically with trial length, presenting a potential computational bottleneck for long-duration recordings.
Conclusion
TNDM represents a significant advancement in neural latent variable modeling. By successfully disentangling behaviorally relevant dynamics without sacrificing neural fit, it provides researchers with a more interpretable and efficient tool for decoding the brain's control of movement. Future work will explore its application to higher-dimensional behaviors, such as muscle activity (EMG) and discrete decision-making processes.
Summary updated