Cross-subject Mapping of Neural Activity with Restricted Boltzmann Machines

Haoming Yang, Marko Angjelichinoski, Suya Wu, Joy Putney, Simon Sponberg, Vahid TarokhbioRxiv 2023 · DOI

TL;DR

This paper introduces a novel framework using Restricted Boltzmann Machines (RBMs) to map neural activity across different subjects, addressing the challenge of generalizing models to new individuals. They propose a more efficient training method for RBMs based on Fisher divergence, which significantly improves cross-subject decoding accuracy compared to existing methods.

What Problem Does This Paper Address?

Generalizing machine learning models trained on one individual's neural data to another individual is a major hurdle in neuroscience. This "cross-subject variability" arises because individuals have unique neural patterns, even when performing the same task. Existing methods often struggle with non-linear neural features (like spiking neurons) and become computationally expensive or require retraining for each new subject. This paper tackles the problem of creating a unified representation space where data from different subjects can be mapped, allowing a decoder trained on one subject (the "source") to accurately interpret data from another subject (the "target") without needing the source's data during testing.

Methods

The core of the proposed solution is a Restricted Boltzmann Machine (RBM), a type of undirected graphical model. Unlike directed generative models (like CVAEs) that can be rigid when adding new subjects, RBMs offer a more flexible, assumption-less approach. The authors leverage the RBM to learn the joint distribution of neural activity across all subjects. During testing, this learned distribution allows them to sample target subject data into the source subject's feature space.

A key innovation is a novel, computationally efficient training technique for RBMs based on Fisher divergence minimization. Traditional RBM training often uses contrastive divergence, which lacks closed-form gradients and requires computationally intensive Gibbs sampling. The Fisher divergence approach, however, allows for closed-form gradient computation, leading to faster and more reliable training.

Datasets

The study uses a dataset of neuromuscular recordings of spike trains from ten muscles controlling wing motion in nine hawk moths (Manduca sexta). This dataset captures a comprehensive motor program driven by six discrete visual stimuli. The data was collected over 20-second recording sessions for each stimulus, segmented into wing strokes (trials).

Key Insights and Ideas

  • RBMs as a Flexible Cross-Subject Mapper: The undirected graphical nature of RBMs makes them inherently more adaptable to new subjects compared to directed models, simplifying the process of transferring knowledge across individuals.
  • Fisher Divergence for Efficient RBM Training: Minimizing Fisher divergence offers a significant computational advantage over contrastive divergence for RBMs, enabling closed-form gradients and eliminating the need for iterative Gibbs sampling during training. This leads to faster and more robust model parameter estimation.
  • Task-Specific Representation is Key: The success of the RBM lies in its ability to learn a non-linear mapping that adapts target data to a task-specific region within the source feature space, rather than just a generic alignment.
  • Scenario I Outperforms Scenario II: Mapping multiple target subjects to a single source (Scenario I) yields better results than mapping a single target to multiple sources (Scenario II). This is intuitively expected, as the latter involves a larger joint source feature vector, potentially leading to more complex inference.

Key Results

The proposed Fisher RBM (RBM-FD) method demonstrated significant improvements in cross-subject decoding accuracy. In Scenario I (one source, all others targets), the RBM-FD achieved up to 95% accuracy in decoding visual stimuli from hawk moth motor programs. This represents a 300% improvement over previous state-of-the-art methods like Hierarchical Wasserstein Alignment (HiWA). Even the RBM trained with contrastive divergence (RBM-CD) outperformed HiWA. The "no transfer" baseline (directly decoding target data with a source-trained decoder) performed poorly, close to random chance, highlighting the necessity of effective cross-subject mapping.

Limitations and Future Work

Limitations: * Asymmetrical Transformation: The RBM's transformation is directional, meaning mapping from subject A to B is not directly equivalent to mapping from B to A without modification. * Limited Similarity Quantification: The current RBM architecture does not easily allow for direct quantification of similarity or distance between subjects or groups of subjects. While conditional entropy can be computed, it's not symmetric. * Lack of Hidden Layer Interpretability: The complex, non-linear nature of the hidden layer in the RBM makes it difficult to interpret how specific features are transformed, unlike algebraic methods that provide explicit mapping matrices.

Future Work: * Deriving the conditional distribution of source features given target features directly from the RBM's joint distribution to avoid noise-like initialization issues. * Developing subject-invariant RBM models that can predict feature representations for entirely unseen target subjects, enhancing generalization. * Applying the RBM model to other neural signal modalities, such as non-invasive EEG signals.

Summary updated

Abstract

Abstract Subject-to-subject variability is a common challenge in generalizing neural data models across subjects. While many methods exist that map one subject to another, it remains challenging to combine many subjects in a computationally efficient manner, especially with features that are highly non-linear such as when considering populations of spiking neurons or motor units. Our objective is to transfer data from one or more target subjects to the data space of one or more source subject(s) such that the neural decoder of the source subject can directly decode the target data when the source(s) is not available during test time. We propose to use the Gaussian-Bernoulli Restricted Boltzmann Machine (RBM); once trained over the entire set of subjects, the RBM allows the mapping of target features on source feature spaces using Gibbs sampling. We also consider a novel computationally efficient training technique for RBMs based on the minimization of the Fisher divergence, which allows the gradients of the RBM to be computed in closed form, in contrast to the more traditional contrastive divergence. We apply our methods to decode turning behaviors from a comprehensive spike-resolved motor program – neuromuscular recordings of spike trains from the ten muscles that control wing motion in an agile flying Manduca sexta. The dataset consists of the comprehensive motor program recorded from nine subjects driven by six discrete visual stimuli. The evaluations show that the target features can be decoded using the source classifier with an accuracy of up to 95% when mapped using an RBM trained by Fisher divergence. Significant Statement In this study, we address the variability of neural data across subjects, which is a significant obstacle in developing models that can generalize across subjects. Our objective is to create a task-specific representation of the target subject signal in the feature space of the source subject. Our proposed RBM architectures achieve highly flexible and accurate cross-subject mapping with few assumptions. Our Fisher RBM improved the previous state of the art method by 300%. Our methods show promise in generalizing features of complex neural datasets across individuals, tuning neural interfaces to subject-specific features, and leveraging data across multiple subjects when experiments are limited in time or completeness.
BibTeX
@misc{yang2023cross,
  author = {Haoming Yang and Marko Angjelichinoski and Suya Wu and Joy Putney and Simon Sponberg and Vahid Tarokh},
  title = {{Cross-subject Mapping of Neural Activity with Restricted Boltzmann Machines}},
  year = {2023},
  doi = {10.1101/2023.04.14.536854},
  howpublished = {bioRxiv},
}
Figure