Abstract
Deep learning tools applied to high-resolution neurophysiological data have significantly progressed, offering enhanced decoding, real-time processing, and readability for practical applications. However, the design of artificial neural networks to analyze neural activity remains a challenge, requiring a delicate balance between efficiency in low-data regimes and the interpretability of the results. To this end, we introduce a novel specialized transformer architecture to analyze single-neuron spiking activity. We test our model on multi electrodes recordings from the dorsal premotor cortex (PMd) of non-human primates while performing a motor inhibition task. The proposed architecture provides a very early prediction of the correct movement direction - no later than 230ms after the Go signal presentation across animals - and can accurately forecast whether the movement will be generated or withheld before a Stop signal, unattended, is actually presented. We also analyze the internal dynamics of the model by computing the predicted correlations between time steps and between neurons at successive layers of the architecture. We find that their evolution mirrors previous theoretical analyses. Overall, our framework provides a comprehensive use case for the practical implementation of deep learning tools in motor control research.
Introduction
High-resolution neurophysiological data recorded directly from the brain is a cornerstone for developing Brain-Machine Interfaces (BMIs). However, translating raw neural spikes into actionable commands for a prosthetic limb or computer cursor is notoriously difficult. The field faces a "Goldilocks" problem: models must be complex enough to capture the intricate dynamics of neural populations, yet efficient enough to function in "low-data regimes" where training examples are limited. Furthermore, clinical and scientific applications require interpretability—researchers need to know why a model makes a specific prediction to ensure it is relying on actual neural planning rather than noise.
Methods and Experimental Design
To address these challenges, the authors introduced the Spatio-Temporal Transformer (ST-T). While standard Transformers are designed for sequences of words, the ST-T is tailored for the "space-time" nature of neural data. The architecture treats neural activity as a grid of "tokens," using alternating blocks of Multi-Head Attention (MHA) to focus on temporal correlations (how a neuron's past influences its future) and spatial correlations (how different neurons interact).
The model was validated using multi-electrode recordings from the dorsal premotor cortex (PMd) of two rhesus macaques. The monkeys performed a "Stop-signal task," where they were trained to reach for a target upon a "Go" signal but to refrain from moving if a subsequent "Stop" signal appeared.
Key Results and Insights
- Early and Accurate Decoding: The ST-T achieved over 90% accuracy in predicting movement direction, reaching 80% accuracy as early as 135ms to 204ms after the Go signal—well before physical movement began.
- Forecasting the "Unseen": Perhaps the most striking result was the model's ability to predict movement inhibition. It could accurately forecast a "Correct Stop" trial before the Stop signal actually appeared, suggesting it captures the internal "preparatory state" or level of caution in the brain.
- Attention as a Scientific Tool: In this framework, attention maps serve as a proxy for neurophysiological synchronization. As data passes through deeper layers, the model builds increasingly complex maps of how neurons synchronize, mirroring established theoretical models of motor planning.
- Robustness Across Subjects: While standard Convolutional Neural Networks (CNNs) often struggle with individual brain variations, the ST-T remained consistently accurate across both subjects tested.
Discussion and Future Work
While the results are promising, the study is currently limited to the PMd and a relatively simple two-direction motor task. Future work involves applying this architecture to more complex, multi-dimensional movements and testing its performance in closed-loop BMI systems. This "relational inference" approach could help scientists discover new types of interactions between neurons that traditional statistical methods might miss, bridging the gap between deep learning and biological functional connectivity.
Summary updated