Mixture of Trajectory Models for Neural Decoding of Goal-Directed Movements

Byron M. Yu, Caleb Kemere, Gopal Santhanam, Afsheen Afshar, Stephen I. Ryu, Teresa H. Meng, Maneesh Sahani, Krishna V. ShenoyJournal of Neurophysiology 2007 · DOI

TL;DR

This paper introduces a novel "Mixture of Trajectory Models" (MTM) approach for decoding intended arm movements from neural data. By combining simpler models that excel in specific movement regimes, MTM decoders achieve significantly higher accuracy than traditional methods, paving the way for more precise brain-computer interfaces.

What Problem Does This Paper Address?

Decoding intended movements from neural signals is crucial for developing advanced prosthetics. A key challenge lies in creating accurate "trajectory models" that capture the complex statistical regularities of human or animal movements. Simple models are computationally cheap but often inaccurate, while complex models can be accurate but too slow for real-time applications. This paper tackles the trade-off between accuracy and computational cost, aiming to develop a decoder that is both precise and efficient enough for real-time prosthetic control.

Methods

The core innovation is the Mixture of Trajectory Models (MTM) framework. Instead of a single, monolithic trajectory model, MTM combines multiple simpler models, each specialized for a particular "regime" of movement (e.g., reaching towards a specific goal). These individual models are probabilistically mixed, allowing the decoder to dynamically switch between them based on the observed neural activity.

The authors employ a recursive Bayesian decoder, which iteratively updates its belief about the arm's state. For the MTM, this involves:

  • Trajectory Models: Each component of the mixture is a linear-Gaussian model, representing the arm's state (position, velocity, acceleration) dynamics. These are trained for each specific reach goal.
  • Observation Model: This model links the neural activity (binned spike counts) to the arm's state, using a cosine tuning model with an exponential to ensure non-negative firing rates.
  • Modal Gaussian Approximation: Since the observation model isn't strictly linear-Gaussian, the authors use a modified Kalman filter with a Gaussian approximation during the measurement update step.
  • Incorporating Prior Goal Information: Crucially, the MTM framework can naturally integrate "delay activity" – neural signals that precede movement and indicate the intended goal. This prior information is used to set the initial weights of the mixture components, improving decoding accuracy.

Datasets

The study utilized neural and behavioral data recorded from two adult male monkeys (Macaca mulatta) performing a delayed center-out reach task. The task involved reaching to one of eight discrete goals. The data sets comprised a large number of successful trials, with neural activity recorded from motor and premotor cortex. The authors also mention preliminary results using a dataset with 16 reach goals.

Key Insights and Ideas

  • The Power of Mixtures: Combining simpler, specialized models (MTM) is a powerful way to create complex, accurate dynamical behaviors that outperform single, general models.
  • Bridging Dynamic and Static Information: MTM effectively integrates two types of neural information: "dynamic" (moment-by-moment movement details) and "static" (intended goal identity from delay activity). This dual-information approach is key to its improved performance.
  • Probabilistic Goal Handling: Instead of a hard "goal decode," MTM uses the probability distribution of potential goals derived from delay activity. This allows the decoder to be more robust to ambiguous or incorrect initial goal information, as peri-movement activity can correct errors.
  • Capturing Movement Variability: MTM's ability to generate different trajectories for the same goal and reflect trial-by-trial variability is a significant advantage over methods that rely on fixed canonical trajectories.

Key Results

  • Reduced Decoding Error: The MTM decoder significantly reduced decoding error compared to simpler models. For monkey G, MTMM (using only peri-movement activity) reduced error by 38% compared to RWM and STM. MTMDM (using both delay and peri-movement activity) further reduced error by 20%. Similar improvements were seen in monkey H (48% and 11%, respectively).
  • No Increase in Running Time: Despite its increased complexity, the MTM decoder's parallelizable nature ensures its running time remains comparable to simpler decoders, making it suitable for real-time applications.
  • Improved Speed Profile Estimation: MTM decoders better captured the characteristic bell-shaped speed profile of actual reaches.
  • Robustness to Unit Count: The performance benefits of MTM were observed across a range of neural unit counts, suggesting its applicability even as implant longevity decreases.
  • Outlier Analysis: The authors identified specific trials where simpler models outperformed MTM due to a "snap-to-component" effect, highlighting areas for future refinement.

Limitations and Future Work

  • Snap-to-Component Effect: In some cases, MTM trajectories transitioned abruptly between component estimates, leading to less smooth decoded paths. This was more pronounced when within-class scatter was small relative to between-class scatter.
  • Novel Goal Decoding: Decoding trajectories to novel goals (not seen during training) requires a denser arrangement of training goals. The authors suggest exploring a ten-by-ten grid of goals.
  • Stereotyped Trajectories: The current data sets had relatively stereotyped trajectories. Future work could explore MTM's ability to capture greater variability in reach speed or curvature.
  • Non-Goal-Directed Movements: The MTM framework's potential application to non-goal-directed movements (e.g., foraging) is suggested, possibly by incorporating random walk mixture components.
  • More Sophisticated Firing Rate Models: With a larger number of goals, more advanced firing rate models may be needed.
  • State Machine for Free Movement: In settings where subjects can freely choose when to reach, a state machine is needed to determine the type of information conveyed by neural activity at each time point.

Summary updated

Abstract

Probabilistic decoding techniques have been used successfully to infer time-evolving physical state, such as arm trajectory or the path of a foraging rat, from neural data. A vital element of such decoders is the trajectory model, expressing knowledge about the statistical regularities of the movements. Unfortunately, trajectory models that both 1) accurately describe the movement statistics and 2) admit decoders with relatively low computational demands can be hard to construct. Simple models are computationally inexpensive, but often inaccurate. More complex models may gain accuracy, but at the expense of higher computational cost, hindering their use for real-time decoding. Here, we present a new general approach to defining trajectory models that simultaneously meets both requirements. The core idea is to combine simple trajectory models, each accurate within a limited regime of movement, in a probabilistic mixture of trajectory models (MTM). We demonstrate the utility of the approach by using an MTM decoder to infer goal-directed reaching movements to multiple discrete goals from multi-electrode neural data recorded in monkey motor and premotor cortex. Compared with decoders using simpler trajectory models, the MTM decoder reduced the decoding error by 38 (48) percent in two monkeys using 98 (99) units, without a necessary increase in running time. When available, prior information about the identity of the upcoming reach goal can be incorporated in a principled way, further reducing the decoding error by 20 (11) percent. Taken together, these advances should allow prosthetic cursors or limbs to be moved more accurately toward intended reach goals.
BibTeX
@article{yu2007mixture,
  author = {Byron M. Yu and Caleb Kemere and Gopal Santhanam and Afsheen Afshar and Stephen I. Ryu and Teresa H. Meng and Maneesh Sahani and Krishna V. Shenoy},
  title = {{Mixture of Trajectory Models for Neural Decoding of Goal-Directed Movements}},
  journal = {Journal of Neurophysiology},
  year = {2007},
  doi = {10.1152/jn.00482.2006},
}
Figure