Abstract
Current brain-computer interfaces primarily decode single motor variables, limiting their ability to support natural, high-bandwidth neural control that requires simultaneous extraction of multiple correlated motor dimensions. We introduce Multi-dimensional Neural Decoding (MND), a task formulation that simultaneously extracts multiple motor variables (direction, position, velocity, acceleration) from single neural population recordings. MND faces two key challenges: cross-task interference when decoding correlated motor dimensions from shared cortical representations, and generalization issues across sessions, subjects, and paradigms. To address these challenges, we propose OrthoSchema, a multi-task framework inspired by cortical orthogonal subspace organization and cognitive schema reuse. OrthoSchema enforces representation orthogonality to eliminate cross-task interference and employs selective feature reuse transfer for few-shot cross-session, subject and paradigm adaptation. Experiments on macaque motor cortex datasets demonstrate that OrthoSchema significantly improves decoding accuracy in cross-session, cross-subject and challenging cross-paradigm generalization tasks, with larger performance improvements when fine-tuning samples are limited. Ablation studies confirm the synergistic effects of all components are crucial, with OrthoSchema effectively modeling cross-task features and capturing session relationships for robust transfer. Our results provide new insights into scalable and robust neural decoding for real-world BCI applications.
Introduction: The Challenge of Multi-dimensional Neural Decoding
For a Brain-Computer Interface (BCI) to support natural, fluid movement—such as controlling a multi-jointed robotic arm—it must simultaneously decode multiple correlated motor variables, including direction, position, velocity, and acceleration. This paper formalizes this requirement as Multi-dimensional Neural Decoding (MND).
Extracting these variables from a single neural population presents two primary hurdles:
- Cross-task interference: Motor variables are often highly correlated and share overlapping cortical representations. Training a model to optimize for one variable (e.g., velocity) can inadvertently degrade the decoding accuracy of another (e.g., position).
- Neural Drift and Distribution Shifts: Neural signals are non-stationary. Factors like electrode micro-movements, biological tissue reactions, or changes in the behavioral paradigm cause signals to drift over time. Consequently, a BCI calibrated on one day may require extensive retraining to remain functional on subsequent days or for different subjects.
Methodology: The OrthoSchema Framework
To address these challenges, the authors introduce OrthoSchema, a multi-task neural network framework inspired by the biological organization of the primate brain. The architecture is designed to disentangle task-relevant information from session-specific noise.
OrthoSchema incorporates three core innovations:
- Orthogonally-Constrained Feature Extractor: The model processes neural spikes through a shared encoder (e.g., RNN or CNN). It applies an orthogonal loss penalty to the latent space, forcing the network to learn independent, non-overlapping representations for different motor tasks. This mimics the "orthogonal subspaces" observed in biological motor cortices, effectively preventing cross-task interference.
- Session Classification Heads: Alongside motor decoding heads, the network includes a dedicated session classification head. During training, this head identifies the specific session or subject of the input data, allowing the model to explicitly isolate and model session-specific "drift."
- Selective Feature Reuse (Transfer Learning): For adaptation to new days or subjects with minimal data (few-shot learning), OrthoSchema identifies weights critical for motor tasks versus those tied to session noise. By discarding session-specific weights and fine-tuning only task-relevant "schemas," the model achieves rapid adaptation without catastrophic forgetting.
Experimental Evaluation
The framework was validated using motor cortex datasets from three rhesus macaques (Subjects C, M, and T). The data covered two distinct paradigms: Center-out (CO) reaching and Random target (RT) reaching.
Key Results
OrthoSchema demonstrated superior performance across three challenging generalization scenarios:
- Cross-Session: In a strict 2-shot setting (only two calibration examples), OrthoSchema improved direction classification accuracy from 72.60% to 80.23% and position decoding ($R^2$) from 0.886 to 0.903.
- Cross-Subject: The model successfully identified universal motor representations, boosting 2-shot direction accuracy from 45.53% to 60.01% when transferring between different monkeys.
- Cross-Paradigm: Even when transitioning from structured Center-out tasks to Random Target tasks, OrthoSchema improved position decoding ($R^2$) from 0.579 to 0.622.
The framework consistently outperformed established baselines, including LFADS, Reduced Rank Regression (RRR), and standard multi-task learning techniques like GradNorm.
Conclusion and Future Directions
OrthoSchema leverages biological principles—orthogonal subspaces and cognitive schema reuse—to create a more robust and scalable BCI decoding framework. By explicitly modeling neural drift and enforcing task independence, it significantly reduces the data requirements for daily BCI calibration. While cross-paradigm transfer remains a difficult frontier, OrthoSchema represents a major step toward universal, "plug-and-play" neural interfaces. Future research will focus on validating these findings in human clinical trials.
Summary updated