Abstract
A grand challenge in brain decoding is to develop algorithms that generalize across multiple subjects and tasks. Here, we developed a new computational framework to minimize negative transfer for domain-adaptive brain decoding by reframing source selection as a mixture model parameter estimation problem, allowing each source subject to contribute through a continuous mixture weight rather than being outright included or excluded. To compute these weights, we developed a novel convex optimization algorithm based on the Generalized Method of Moments. By using model performance metrics as the generalized moment functions in our GMM optimization, our algorithm also provides theoretical guarantees that the mixture weights are an optimal approximation of the importance weights that underlie domain adaptation theory. When tested on a large-scale brain decoding dataset (n=105 subjects), our new mixture model weighting framework achieved state-of-the-art performance—increasing accuracy up to 2.5% over baseline fine-tuning, double the performance gain compared to previous research in supervised source selection. Notably, these improvements were achieved using significantly less training data (i.e., 62% smaller effective sample sizes), suggesting that our performance gains stem from reduced negative transfer. Collectively, this research advances toward a more principled and generalizable brain decoding framework, laying the mathematical foundation for scalable brain-computer interfaces and other applications in computational neuroscience.
Introduction: The Challenge of Negative Transfer
Brain decoding—the process of predicting behavior or cognitive states from neural activity—is notoriously difficult because every brain is unique. In most deep learning applications, the prevailing wisdom is that "more data is better." However, in brain decoding, researchers often encounter negative transfer. Because brain activity is so subject-specific, training a model on a large, diverse group of people can actually degrade its performance on a specific target individual.
Current solutions typically involve "source selection," where researchers attempt to pick the $k$ most similar people to the target subject. This is often a binary "in or out" choice based on heuristics. We argue that this approach is too rigid and fails to capture the nuanced similarities between different subjects.
Methods: A Mixture Model Framework
We reframe source selection as a mixture model parameter estimation problem. Instead of deciding whether to include a subject or not, our framework assigns each source subject a continuous "mixture weight" ($\hat{\pi}$).
To find these weights, we developed a novel convex optimization algorithm based on the Generalized Method of Moments (GMM). The process uses model performance metrics (like cross-entropy loss) as the "moments" and minimizes the square error between the generalized moments of the weighted source subjects and the target subject. This ensures the weighted source data distribution closely matches the target distribution.
The training occurs in three distinct phases of Curriculum Adaptation:
- Global Pre-training: Training on all subjects with equal weights.
- Weighted Domain Adaptation: Fine-tuning the model using the calculated mixture weights to emphasize relevant source data.
- Subject-Specific Fine-tuning: A final polish using only the target subject's small data set.
Key Insights: Quality Over Quantity
The move from binary inclusion to continuous weighting allows the model to "softly" include data, which is more theoretically aligned with importance sampling in domain adaptation.
Testing on the PhysioNet BCI2000 dataset (105 subjects, 64-channel EEG) revealed that quality matters far more than quantity. Our framework achieved state-of-the-art results while using an Effective Sample Size (ESS) that was 62% smaller than traditional methods. Furthermore, we found that optimal weight distributions varied wildly between subjects; a "one-size-fits-all" $k$ value for source selection is fundamentally suboptimal.
Key Results and Performance
- Accuracy Boost: The mixture model framework achieved a 2.5% increase in accuracy over baseline subject-specific fine-tuning.
- Outperforming Benchmarks: This result doubled the performance gains seen in previous state-of-the-art source selection methods (~1.3% boost).
- Theoretical Grounding: Unlike heuristic methods, this approach provides a mathematical guarantee that the mixture weights are an optimal approximation of the importance weights required for domain adaptation.
Limitations and Future Work
While the framework significantly reduces negative transfer, it does not explicitly target overfitting, which remains a challenge in neural decoding. Future work may explore more granular "class-level" weighting (e.g., weighting a subject's "left hand" data differently than their "right hand" data) and combining this weighting framework with latent feature alignment techniques to further improve cross-subject generalization.