Consistency of EEG source localization and connectivity estimates

Keyvan Mahjoory, Vadim V. Nikulin, Loïc Botrel, Klaus Linkenkaer‐Hansen, Marco Fato, Stefan Haufe

TL;DR

This paper tackles the challenge of reliably estimating brain activity and its connections from EEG data, which is complicated by the "EEG inverse problem." They found that while localizing brain sources is relatively stable across different analysis methods, estimating how brain regions communicate (connectivity) is highly variable, especially for directed (effective) connectivity. This highlights the need for caution and multiple validation steps when interpreting EEG connectivity findings.

What Problem Does This Paper Address?

Electroencephalography (EEG) offers excellent temporal resolution for studying brain activity, but its spatial resolution is poor. To understand where brain activity originates and how different regions interact, researchers use "source localization" to estimate the brain sources from scalp EEG signals. However, this is an "ill-posed problem," meaning there isn't a single, unique solution. The reconstructed sources and, consequently, any derived connectivity measures, can vary significantly depending on numerous choices made during the analysis pipeline. These choices include:

  • Anatomical templates: The underlying brain model used (e.g., ICBM152, Colin27).
  • Electrical models: How the electrical conductivity of the head tissues is modeled (e.g., Boundary Element Method - BEM, Finite Element Method - FEM, Spherical Harmonics Expansions - SHE).
  • Inverse methods: The algorithms used to estimate source activity from sensor data (e.g., Weighted Minimum Norm Estimate - WMNE, Linearly Constrained Minimum Variance - LCMV, eLORETA).
  • Software implementations: Different toolboxes (Brainstorm, Fieldtrip, Berlin toolbox) can implement these methods with subtle variations.

This variability makes it difficult to reproduce findings and trust the reliability of EEG-based brain connectivity studies. The paper aims to systematically assess this variability and understand which factors contribute most to it.

Methods

The researchers conducted a comprehensive comparison of EEG source localization and connectivity estimation using various combinations of parameters. They employed:

  • Two anatomical templates: ICBM152 and Colin27.
  • Three electrical models: BEM, FEM, and SHE.
  • Three inverse methods: WMNE, eLORETA, and LCMV.
  • Three software implementations: Brainstorm, Fieldtrip, and their own "Berlin toolbox."

This resulted in 14 distinct "pipelines" for source reconstruction. They analyzed resting-state EEG data, focusing on alpha-band oscillations (8-13 Hz), which are prominent and relatively stable in topography.

Key methodological steps included:

  • Spatio-spectral decomposition (SSD): Used to remove noise and enhance the alpha-band signal.
  • Forward modeling: Calculating the lead fields that describe how electrical signals propagate from brain sources to EEG sensors.
  • Inverse modeling: Estimating source activity from sensor data using the chosen inverse methods.
  • Connectivity measures:
    • Functional Connectivity (FC): Measured using the imaginary part of coherency (iCOH), which is considered robust to source mixing.
  • Effective Connectivity (EC): Measured using the Phase Slope Index (PSI), which estimates directed interactions.

Consistency was quantified by calculating Pearson correlations between the results obtained from different pipelines, datasets, and participants.

Datasets

The study utilized resting-state EEG data from two independent experiments:

  • Fasor data (FD): N=30 participants, recorded with 128 electrodes (reduced to 64 for analysis).
  • Würzburg data (WD): N=35 participants, recorded with 64 electrodes.

A subset of 49 electrodes common to both datasets was used for the analysis to ensure comparability.

Key Insights and Ideas

  • Source localization is more stable than connectivity: While different pipelines can produce slightly different maps of where brain activity is located, these maps are generally more consistent than the estimated connections between brain regions.
  • Effective connectivity is the least consistent: Estimating directed interactions between brain regions (effective connectivity) showed the most variability across different analysis choices. This suggests that our understanding of directed brain communication from EEG is particularly sensitive to methodological decisions.
  • Inverse method matters most: The choice of inverse method (e.g., LCMV vs. eLORETA/WMNE) had a larger impact on the results than the choice of anatomical template or electrical model. Specifically, LCMV beamformers produced more focal source localization results compared to distributed methods like eLORETA and WMNE.
  • Software implementation also introduces variability: Even when using the same underlying methods, different software packages can lead to noticeable differences in results.
  • Consistency decreases with comparison scope: Results were most consistent when comparing different pipelines on the same data, less consistent when comparing across participants within a study, and least consistent when comparing across different studies.
  • Resting-state data might amplify variability: The authors suggest that the inherent variability in individual "resting-state phenotypes" might contribute to the lower consistency observed for connectivity measures, as participants' thoughts and internal states can vary significantly without a task.

Key Results

  • Source localization: Generally stable across pipelines, with correlations often exceeding 0.75. LCMV produced more focal SNR maps than eLORETA/WMNE.
  • Functional connectivity (FC): More consistent than effective connectivity, with average correlations around 0.70-0.97 depending on the parameter varied.
  • Effective connectivity (EC): The least consistent, with average correlations as low as 0.33-0.35 when varying software or inverse methods.
  • Impact of parameters:
    • Forward models: Had a relatively small impact on consistency (r ≈ 0.99 for LOC).
  • Inverse methods: Induced considerable variability (r ≈ 0.35-0.77 for LOC, r ≈ 0.35-0.61 for FC, r ≈ 0.33-0.35 for EC).
  • Software packages: Also induced considerable variability (r ≈ 0.70 for FC, r ≈ 0.33 for EC).
  • Between-study consistency: Grand-average results showed high consistency (r ≈ 0.99 for LOC, r ≈ 0.82 for FC), but this dropped significantly at the single-participant level (r ≈ 0.77 for LOC, r ≈ 0.48 for FC).

Limitations and Future Work

  • Limited number of electrodes: The study used a moderate number of electrodes (49), and the authors acknowledge that higher-density EEG data might yield different results.
  • Resting-state data: The use of resting-state data, while ideal for studying intrinsic brain states, may contribute to the observed variability in connectivity due to individual differences in cognitive states. Future work could explore consistency across different experimental conditions.
  • Need for simulations with ground truth: The authors emphasize the need for future studies using simulated data with known ground truth to rigorously assess the accuracy and biases of different pipelines, especially for time-series dynamics relevant to connectivity.
  • Comparison with MEG: A comparison of EEG results with those from MEG would be valuable.
  • Higher-level parameters: The study focused on variations in core parameters. Future work could investigate the impact of higher-level parameters like regularization constants and depth-compensation strategies.
  • Recommendation for practice: The authors strongly encourage researchers to use multiple source imaging procedures and to report their findings with an awareness of the inherent variability and uncertainty.

Summary updated

Abstract

Abstract As the EEG inverse problem does not have a unique solution, the sources reconstructed from EEG and their connectivity properties depend on forward and inverse modeling parameters such as the choice of an anatomical template and electrical model, prior assumptions on the sources, and further implementational details. In order to use source connectivity analysis as a reliable research tool, there is a need for stability across a wider range of standard estimation routines. Using resting state EEG recordings of N=65 participants acquired within two studies, we present the first comprehensive assessment of the consistency of EEG source localization and functional/effective connectivity metrics across two anatomical templates (ICBM152 and Colin27), three electrical models (BEM, FEM and spherical harmonics expansions), three inverse methods (WMNE, eLORETA and LCMV), and three software implementations (Brainstorm, Fieldtrip and our own toolbox). Source localizations were found to be more stable across reconstruction pipelines than subsequent estimations of functional connectivity, while effective connectivity estimates where the least consistent. All results were relatively unaffected by the choice of the electrical head model, while the choice of the inverse method and source imaging package induced a considerable variability. In particular, a relatively strong difference was found between LCMV beamformer solutions on one hand and eLORETA/WMNE distributed inverse solutions on the other hand. We also observed a gradual decrease of consistency when results are compared between studies, within individual participants, and between individual participants. In order to provide reliable findings in the face of the observed variability, additional simulations involving interacting brain sources are required. Meanwhile, we encourage verification of the obtained results using more than one source imaging procedure.
BibTeX
@article{mahjoory2017consistency,
  author = {Keyvan Mahjoory and Vadim V. Nikulin and Loïc Botrel and Klaus Linkenkaer‐Hansen and Marco Fato and Stefan Haufe},
  title = {{Consistency of EEG source localization and connectivity estimates}},
  journal = {NeuroImage},
  year = {2017},
  doi = {10.1016/j.neuroimage.2017.02.076},
}
Figure