Abstract: How does the motor cortex combine simple movements (such as single finger flexion/extension) into complex movements (such as hand gestures, or playing the piano)? To address this question, motor cortical activity was recorded using intracortical multi-electrode arrays in two male people with tetraplegia as they attempted single, pairwise and higher-order finger movements. Neural activity for simultaneous movements was largely aligned with linear summation of corresponding single finger movement activities, with two violations. First, the neural activity exhibited normalization, preventing a large magnitude with an increasing number of moving fingers. Second, the neural tuning direction of weakly represented fingers changed significantly as a result of the movement of more strongly represented fingers. These deviations from linearity resulted in non-linear methods outperforming linear methods for neural decoding. Simultaneous finger movements are thus represented by the combination of individual finger movements by pseudo-linear summation.
Introduction
The human hand is a marvel of engineering, capable of more than 20 degrees of freedom. While neuroscientists have a good grasp of how the brain represents the movement of a single finger, it has remained unclear how the motor cortex "composes" these individual movements into complex gestures or tasks like playing the piano. Does the brain have dedicated "gesture neurons," or does it simply add single-finger signals together? Understanding this "neural geometry"—the way populations of neurons organize information in high-dimensional space—is critical for developing brain-computer interfaces (BCIs) that can restore natural, fluid hand function to people with paralysis.
Methods and Experimental Design
The study involved two participants with tetraplegia (T5 and T11) who had 96-channel Utah micro-electrode arrays implanted in the "hand knob" area of the premotor cortex. Researchers recorded neural activity while participants attempted three types of movements cued by a 3D animated hand:
- Single-finger movements: Flexion or extension of one finger at a time.
- Natural gestures: 38 movements, including the ASL alphabet and common signs.
- Combinatorial movements: 80 unique combinations where four finger groups were independently cued to flex, extend, or remain idle.
The team used Targeted Dimensionality Reduction (TDR) to visualize the data and compared three mathematical models: a standard Linear Model, a Quadratic Model, and a Linear-Nonlinear (Pseudo-linear) Model using a saturating tanh function.
Key Insights: The Pseudo-linear Model
The researchers discovered that neural activity for multi-finger movements is largely aligned with the linear sum of single-finger activities, suggesting the brain reuses "building blocks" of individual finger representations. However, a pure linear sum (Hypothesis 1 in the figure) fails because neural activity does not grow indefinitely. Instead, the brain exhibits normalization (Hypothesis 3), keeping the overall magnitude of neural activity relatively constant regardless of how many fingers are moving.
This normalization leads to "tuning shifts" in weakly represented fingers. When a "strong" finger (like the thumb) moves, it "pushes" the representation of weaker fingers (like the middle finger) in neural space. This pseudo-linear summation—adding signals then applying a saturating curve—explained the data as well as complex quadratic models but with far fewer parameters, suggesting the interaction is a byproduct of physical neural constraints rather than complex computation.
Results and Decoding Performance
- Model Accuracy: The pseudo-linear model captured 56% of the variance in neural activity and significantly outperformed the linear model in predicting activity magnitude.
- Decoding Challenges: A linear decoder achieved 76% accuracy on natural gestures but struggled with complex combinatorial movements (40% accuracy), highlighting the impact of non-linear deviations.
- Generalization: Linear decoders trained on isolated movements performed poorly when tested on multi-finger contexts for the index and middle fingers (dropping to ~60% accuracy), while the thumb remained robust (~76%).
Limitations and Future Directions
This study focused on static gestures during the "hold" period. Real-world hand use is dynamic and continuous; future work is needed to see if pseudo-linear summation holds during fluid transitions. Additionally, because the participants have tetraplegia, the study lacks the "closed-loop" component of sensory feedback. Investigating how proprioception interacts with this neural geometry remains a promising frontier for BCI research.
Summary updated