Diversity encouraged learning of unsupervised LSTM ensemble for neural activity video prediction

Yilin Song, Jonathan Viventi, Yao Wang

Abstract: Being able to predict the neural signal in the near future from the current and previous observations has the potential to enable real-time responsive brain stimulation to suppress seizures. We have investigated how to use an auto-encoder model consisting of LSTM cells for such prediction. Recognizing that there exist multiple activity pattern clusters, we have further explored to train an ensemble of LSTM models so that each model can specialize in modeling certain neural activities, without explicitly clustering the training data. We train the ensemble using an ensemble-awareness loss, which jointly solves the model assignment problem and the error minimization problem. During training, for each training sequence, only the model that has the lowest reconstruction and prediction error is updated. Intrinsically such a loss function enables each LTSM model to be adapted to a subset of the training sequences that share similar dynamic behavior. We demonstrate this can be trained in an end-to-end manner and achieve significant accuracy in neural activity prediction.

Introduction

Epilepsy affects millions worldwide, with approximately one-third of patients failing to respond to conventional medication. Responsive brain stimulation (RBS) offers a promising alternative, utilizing implanted devices to detect and suppress seizures via electrical pulses. However, current RBS systems are often reactive. To achieve proactive suppression, devices must predict neural activity in the near future (milliseconds). This task is challenging due to the highly non-linear and complex spatial patterns of neural signals, typically captured via high-resolution micro-electrocorticography ($\mu$ECoG).

Methods: LSTM Ensemble with Multiple Choice Learning

The researchers treat $\mu$ECoG data as a video sequence, where each frame represents spatial neural activity. They employ an LSTM encoder-decoder-predictor framework, enhanced by an ensemble approach using Multiple Choice Learning (MCL).

Instead of a single large model, the system uses an ensemble of smaller LSTM networks. During training, an "ensemble-awareness loss" is used: for any input sequence, only the model producing the lowest reconstruction and prediction error is updated. This "winner-take-all" strategy encourages diversity, allowing individual models to specialize in specific neural dynamics without explicit data labeling.

Delay maps (I, II, III) showing different neural activity patterns and their respective time scales.

To handle hardware limitations like missing channels, the team used graph filtering and converted signals into "delay maps" to better visualize wave propagation. The dataset was derived from a feline model of epilepsy using an 18x20 electrode array.

Unsupervised Specialization

A key finding is that the MCL approach acts as an implicit clustering mechanism. The ensemble naturally partitions the training data, with specific models becoming "experts" in distinct physical wave behaviors, such as spirals or plane waves, without prior knowledge of these patterns.

The ensemble naturally clusters neural activity into distinct regimes, with different models specializing in specific wave dynamics.

At test time, the system selects the best expert by evaluating reconstruction error on the immediate past sequence or using a trained MLP classifier to predict which specialized LSTM will perform best.

Results

The ensemble approach significantly outperformed single-model baselines. While a single large LSTM (3000 nodes) suffered from overfitting (25.81 PSNR), the MCL ensemble achieved a Peak Signal-to-Noise Ratio (PSNR) of 31.28 using a classifier, approaching the "Oracle" ceiling of 32.26. Qualitatively, the specialized models maintained the sharpness of neural wave fronts, whereas single-model approaches tended to produce blurred, averaged-out predictions.

Conclusion

By encouraging model diversity through unsupervised ensemble learning, this research demonstrates a robust method for high-resolution neural activity prediction. This specialization allows for more accurate forecasting of complex brain signals, paving the way for more effective, proactive responsive brain stimulation therapies.

Summary updated

Abstract

Being able to predict the neural signal in the near future from the current and previous observations has the potential to enable real-time responsive brain stimulation to suppress seizures. We have investigated how to use an auto-encoder model consisting of LSTM cells for such prediction. Recog- nizing that there exist multiple activity pattern clusters, we have further explored to train an ensemble of LSTM mod- els so that each model can specialize in modeling certain neural activities, without explicitly clustering the training data. We train the ensemble using an ensemble-awareness loss, which jointly solves the model assignment problem and the error minimization problem. During training, for each training sequence, only the model that has the lowest recon- struction and prediction error is updated. Intrinsically such a loss function enables each LTSM model to be adapted to a subset of the training sequences that share similar dynamic behavior. We demonstrate this can be trained in an end- to-end manner and achieve significant accuracy in neural activity prediction.
BibTeX
@misc{song2016diversity,
  author = {Yilin Song and Jonathan Viventi and Yao Wang},
  title = {{Diversity encouraged learning of unsupervised LSTM ensemble for neural activity video prediction}},
  year = {2016},
  eprint = {1611.04899},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1611.04899},
}
Figure