Abstract
Relationships between neuroimaging measures and behavior provide important clues about brain function and cognition in healthy and clinical populations. While electroencephalography (EEG) provides a portable, low-cost measure of brain dynamics, it has been somewhat underrepresented in the emerging field of model-based inference. This article highlights the utility of linking EEG and behavior, emphasizing approaches that move beyond traditional event-related potential (ERP) components. We review methods for single-trial feature extraction—including user-defined filters, data-driven filters, and deep learning—and cognitive models like the drift diffusion model (DDM) and reinforcement learning (RL). Finally, we discuss integration strategies such as statistical models, data-driven joint models, and hierarchical Bayesian modeling (HBM).
Introduction: The Limitations of Averaging
For decades, the standard for EEG analysis has been the Event-Related Potential (ERP). By averaging EEG signals across hundreds of trials, researchers aim to wash out noise and reveal consistent "peaks" like the P300 or N200. However, this approach is fundamentally limited. Averaging discards the trial-to-trial variability that likely reflects meaningful fluctuations in attention, learning, and decision-making. Furthermore, traditional ERP components are often nebulously defined and do not always map cleanly onto psychological constructs. There is a significant gap in "model-based inference" for EEG, where neural data is directly integrated with computational models of behavior to explain how the brain produces specific actions.
A Hierarchy of Integration Methods
Bridging the gap between raw EEG data and behavioral outcomes requires a structured approach to processing and modeling.
1. Feature Extraction and Signal Enhancement
To move beyond averaged components, researchers must enhance the signal within single trials to extract features relevant to the experiment:
- User-Defined Filters: Techniques like wavelet decomposition allow for time-frequency analysis, capturing biologically relevant oscillations (e.g., theta or alpha bands) that vary over time.
- Data-Driven Filters: Blind Source Separation (BSS), such as Independent Component Analysis (ICA), helps disentangle neural signals from non-brain artifacts like eye blinks or muscle tension.
- Deep Learning: Utilizing Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) to learn abstract, non-linear representations of EEG data without manual feature engineering.
2. Cognitive Modeling of Behavior
Instead of relying solely on raw behavioral metrics like simple accuracy or reaction time, researchers use cognitive models to derive latent measures of cognitive function:
- Drift Diffusion Model (DDM): This model decomposes reaction time and accuracy into latent variables like "drift rate" (the speed of evidence accumulation) and "threshold" (the level of decision caution).
- Reinforcement Learning (RL): RL models extract variables such as "reward prediction error" (RPE), allowing researchers to track how the brain updates expectations on a trial-by-trial basis.
3. Data Fusion and Joint Modeling
The final step is integrating these neural and behavioral features using various statistical approaches:
- Single-Trial Regression: Correlating neural features directly with behavioral measures within a single subject.
- Representational Similarity Analysis (RSA): Mapping different data types (EEG, fMRI, behavior) into a common "dissimilarity space" to compare them directly.
- Hierarchical Bayesian Models (HBMs): The most sophisticated approach, where neural and behavioral data are modeled simultaneously to jointly estimate latent cognitive parameters.
Key Insights and Results
The shift toward integrated single-trial analysis has yielded significant quantitative and qualitative successes:
- Variability as a Feature: Trial-by-trial fluctuations in EEG are not just noise; they correlate with changes in cognitive states that averaged ERPs completely miss.
- Targeting Latent Variables: Neural signals often correlate more strongly with the hidden parameters of a cognitive model (like a learning rate) than with raw behavior (like a simple "hit" or "miss").
- Visualization and Prediction: Techniques like t-SNE can project high-dimensional single-trial data into 2D space, showing clear clusters for different stimuli. Furthermore, using HBMs, researchers have shown that single-trial N200 amplitudes can successfully predict trial-to-trial differences in the "drift rate" of decision-making.
- Neural Signatures of Learning: RL models have successfully linked mid-frontal EEG activity to reward prediction errors, providing a millisecond-by-millisecond account of how we learn from mistakes.
Challenges and Future Directions
Despite the promise of these methods, several limitations remain. Single-trial EEG is inherently noisy, making the success of these methods heavily dependent on the quality of initial filtering and source separation. The authors also warn of the "Horse" phenomenon, where deep learning models achieve high accuracy by inadvertently training on artifacts (like eye movements) rather than brain activity.
Future work should focus on developing more "generative" models that don't just correlate EEG with behavior but explain the biological mechanism by which cortical dynamics cause the observed latent cognitive variables. Additionally, the field needs larger, shared repositories of single-trial data to fully leverage the power of deep learning and complex Bayesian models. By moving beyond averaged components, researchers can uncover the dynamic, trial-by-trial mechanisms that drive human cognition.
Summary updated