Stabilizing brain-computer interfaces through alignment of latent dynamics

Brianna M. Karpowicz, Yahia H Ali, Lahiru N. Wimalasena, Andrew R. Sedler, Mohammad Reza Keshtkaran, Kevin Bodkin, Xuan Ma, Lee E. Miller, Chethan PandarinathbioRxiv 2022 · DOI

TL;DR

Brain-computer interfaces (BCIs) can restore function to people with paralysis, but their performance degrades over time due to unstable neural recordings. This paper introduces NoMAD, a novel unsupervised method that leverages the dynamics of neural activity to stabilize BCI decoding for months, eliminating the need for frequent recalibration.

What Problem Does This Paper Address?

Intracortical brain-computer interfaces (iBCIs) translate brain activity into control signals for external devices, offering a pathway to restoring motor function for individuals with paralysis. However, a major hurdle to their widespread clinical adoption is the instability of neural recordings. Over time, factors like electrode shifts, tissue response, and electrode failure cause changes in the recorded neural signals. This leads to a "non-stationary" input to the BCI decoder, meaning the relationship between brain activity and intended movement changes. Consequently, BCIs require frequent, supervised recalibration, which interrupts daily use and is burdensome for users. Existing methods for stabilizing BCIs often rely on identifying underlying "latent manifolds" in neural activity, but they typically treat each moment in time independently, neglecting the temporal dynamics of neural populations. This paper aims to address this by incorporating these dynamics into the stabilization process.

Methods

The core innovation is NoMAD (Nonlinear Manifold Alignment with Dynamics). NoMAD builds upon existing manifold-based BCI approaches but crucially incorporates recurrent neural network (RNN) models of neural dynamics.

Here's a breakdown of the key components:

  • Latent Factor Analysis via Dynamical Systems (LFADS): This is the underlying model used to capture the neural population dynamics. LFADS uses RNNs to model the temporal evolution of neural activity, learning a low-dimensional representation (the "manifold") and its dynamics.
  • Unsupervised Alignment: The key novelty is how NoMAD adapts to new data (Day K) from the initial training data (Day 0). Instead of requiring new labeled behavioral data for recalibration, NoMAD uses an "alignment network" to transform the Day K neural data to match the Day 0 manifold and dynamics. This alignment is performed unsupervised, meaning it doesn't need explicit behavioral labels from Day K.
  • Distribution Alignment: The unsupervised alignment process aims to minimize the difference between the distributions of the learned latent states (from LFADS) on Day 0 and Day K. This is achieved by minimizing the Kullback-Leibler (KL) divergence between these distributions, focusing on matching the first and second-order statistical moments.
  • Recurrent Neural Network Dynamics: By modeling the temporal evolution of neural activity, NoMAD can better account for how neural states change over time, leading to more robust and stable decoding.

Datasets

The paper evaluates NoMAD on two primary datasets collected from monkeys performing motor tasks:

  1. Isometric Force Task: Data collected from monkey primary motor cortex (M1) over 95 days. This dataset involves generating wrist forces to control a cursor.
  2. Unloaded Reaching Task: Data collected from monkey M1 over 38 days, involving center-out reaching movements using a planar manipulandum.

These datasets were used for both training the initial Day 0 model and for evaluating the performance of NoMAD and comparison methods across sessions spanning weeks to months.

Key Insights and Ideas

  • Dynamics are Crucial for Stability: The paper strongly suggests that simply aligning static neural manifolds isn't enough for long-term BCI stability. Incorporating the temporal dynamics of neural activity is key to compensating for recording instabilities.
  • Unsupervised Alignment is Feasible and Effective: NoMAD demonstrates that it's possible to achieve significant stabilization without requiring users to provide new labeled data, a major step towards practical clinical applications.
  • Long-Term Stability Achieved: The results show that NoMAD can maintain stable decoding performance for months, far exceeding previous methods. This implies a significant reduction in the need for frequent recalibration.
  • Generalizability Across Tasks: NoMAD's effectiveness was demonstrated on two distinct motor tasks (isometric force and reaching), suggesting its potential applicability to a range of BCI applications.

Key Results

  • Isometric Force Task: NoMAD achieved a median R² of 0.91 with minimal variability and no decoding failures over a 3-month period. This is a significant improvement compared to previous methods like Aligned FA (median R²=0.59) and ADAN (median R²=0.65), which showed more variability and decoding failures. The half-life of decoding performance for NoMAD was 209.1 days, compared to 45.1 days for Aligned FA and 76.7 days for ADAN.
  • Unloaded Reaching Task: NoMAD achieved a median R² of 0.77 with no decoding failures and less variability over 5 weeks. Again, this outperformed Aligned FA (median R²=0.17) and ADAN (median R²=0.29), which struggled with this more challenging dataset. The half-life for NoMAD was 61.4 days.
  • Visualizations: Manifold visualizations showed that NoMAD successfully aligned the neural activity from later sessions (Day K) to the structure of the initial session (Day 0), even after significant time had passed.

Limitations and Future Work

  • Computational Demands: The authors acknowledge that the NoMAD alignment process, like most neural network training, is computationally demanding. For implanted devices with severe power constraints, further optimization is needed.
  • Unsupervised Constraint: The current work strictly adheres to unsupervised alignment, meaning no behavioral information from Day K is used. In practical applications, relaxing this constraint by incorporating available behavioral labels could further enhance performance.
  • Learning New Skills: The current approach assumes a stable relationship between neural activity and behavior. If subjects are learning entirely new skills, which could lead to manifold-level changes, stabilization might need to be run at shorter intervals or adapted to account for learning.
  • Multiple Manifolds: The current tests focus on single behaviors. Future work could explore how to handle scenarios where different behaviors might occupy distinct neural manifolds.
  • Generalization to Other Modalities: While the underlying LFADS model has been applied to ECoG data, further investigation is needed to confirm NoMAD's generalization to other BCI recording modalities.

Summary updated

Abstract

Abstract Intracortical brain-computer interfaces (iBCIs) restore motor function to people with paralysis by translating brain activity into control signals for external devices. In current iBCIs, instabilities at the neural interface result in a degradation of decoding performance, which necessitates frequent supervised recalibration using new labeled data. One potential solution is to use the latent manifold structure that underlies neural population activity to facilitate a stable mapping between brain activity and behavior. Recent efforts using unsupervised approaches have improved iBCI stability using this principle; however, existing methods treat each time step as an independent sample and do not account for latent dynamics. Dynamics have been used to enable high performance prediction of movement intention, and may also help improve stabilization. Here, we present a platform for Nonlinear Manifold Alignment with Dynamics (NoMAD), which stabilizes iBCI decoding using recurrent neural network models of dynamics. NoMAD uses unsupervised distribution alignment to update the mapping of nonstationary neural data to a consistent set of neural dynamics, thereby providing stable input to the iBCI decoder. In applications to data from monkey motor cortex collected during motor tasks, NoMAD enables accurate behavioral decoding with unparalleled stability over weeks-to months-long timescales without any supervised recalibration.
BibTeX
@misc{karpowicz2022stabilizing,
  author = {Brianna M. Karpowicz and Yahia H Ali and Lahiru N. Wimalasena and Andrew R. Sedler and Mohammad Reza Keshtkaran and Kevin Bodkin and Xuan Ma and Lee E. Miller and Chethan Pandarinath},
  title = {{Stabilizing brain-computer interfaces through alignment of latent dynamics}},
  year = {2022},
  doi = {10.1101/2022.04.06.487388},
  howpublished = {bioRxiv},
}
Figure