How reliable are MEG resting-state connectivity metrics?

Giles L. Colclough, Mark W. Woolrich, Prejaas Tewarie, Matthew J. Brookes, Andrew J. Quinn, Stephen M. SmithNeuroImage 2016 · DOI

TL;DR

This paper investigates the reliability of various MEG-based resting-state connectivity metrics. The authors find that many popular methods are heavily influenced by spatial leakage, leading to artificially high consistency. They recommend amplitude envelope correlation and partial correlation, especially when corrected for spatial leakage, as the most robust options for reliable resting-state network analysis.

What Problem Does This Paper Address?

Magnetoencephalography (MEG) offers excellent temporal and spectral resolution for studying brain activity, making it a valuable tool for investigating resting-state functional connectivity. However, unlike fMRI, MEG data is prone to "spatial leakage," where signals from one brain region can contaminate the measurements of neighboring regions. This leakage can create spurious connections, making it difficult to accurately assess true functional interactions. Furthermore, there's a wide array of analytical methods for estimating connectivity from MEG data, but little guidance on which ones are reliable and reproducible. Researchers often pick methods without a clear understanding of their limitations, potentially leading to misleading conclusions. This paper aims to address this by systematically evaluating the reliability and reproducibility of commonly used MEG resting-state connectivity metrics.

Methods

The researchers employed a multi-faceted approach to assess the reliability of 12 different connectivity metrics.

  • Source Localization: They used a linearly constrained minimum variance scalar beamformer to estimate the location of magnetic sources in the brain. This method was chosen for its relative robustness to field spread.
  • Parcellation: The brain was divided into 39 functionally defined regions of interest (ROIs) based on fMRI data from the Human Connectome Project (HCP). For each ROI, a single time-course was extracted as the first principal component of its activity.
  • Connectivity Metrics: A diverse set of 12 metrics were tested, encompassing:
  • Phase-based methods: Phase-locking value (PLV), phase lag index (PLI), weighted phase lag index (wPLI), phase slope index (PSI), and mutual information between phases (MI).
  • Spectral coherence methods: Absolute coherence (Coh), imaginary coherency (IMC), partial coherence (PCoh), and imaginary partial coherency (IMPC).
  • Amplitude coupling methods: Amplitude envelope correlation (AEC) and partial amplitude envelope correlation (PAEC).
  • Auto-regressive modeling: Partial directed coherence (PDC).
  • Spatial Leakage Correction: A key aspect of their methodology was the application of a symmetric orthogonalization procedure to mitigate spatial leakage. This correction was applied to specific metrics known to be susceptible.
  • Reliability Assessment: Three primary criteria were used to evaluate reliability:
  • Group-level repeatability: How consistent are the estimated group-level network matrices when the dataset is split in half?
  • Within-subject consistency: How reproducible are network estimates from repeated MEG recordings of the same individual?
  • Between-subject consistency: How similar are network estimates across different individuals?

Datasets

The study utilized publicly available resting-state MEG data from the Human Connectome Project (HCP). Specifically, they used data from 61 subjects who had undergone three consecutive resting-state recording sessions. This dataset was chosen for its stringent preprocessing, large sample size of healthy young adults, and repeated measurements, which are ideal for assessing reliability.

Key Insights and Ideas

  • Spatial Leakage is a Major Confound: The most striking finding is the profound impact of spatial leakage. Many connectivity metrics, when uncorrected, show artificially high consistency and repeatability because they are picking up spurious correlations caused by signals bleeding between regions.
  • "Leakage-Prone" Metrics Show High (False) Consistency: Metrics like absolute coherence and amplitude envelope correlation, which are sensitive to spatial leakage, exhibit very high correlations when comparing different halves of the dataset or different sessions. This is not a sign of true reliability but rather an artifact of the leakage.
  • Leakage Correction is Crucial: Applying a multivariate correction for spatial leakage significantly reduces the spurious consistency observed in susceptible metrics. This highlights the necessity of such corrections for accurate interpretation.
  • Phase-Based Metrics are Less Reliable: Metrics like PLI, wPLI, and imaginary coherency, which are designed to be less sensitive to zero-lag leakage, generally show poor test-retest reliability, especially at the single-subject level. This suggests they are either inherently noisy or require longer scan times to achieve sufficient signal-to-noise ratio.
  • Amplitude Envelope Correlation and Partial Correlation Shine: Amplitude envelope correlation (AEC), particularly when corrected for spatial leakage, and partial correlation (especially when leakage-corrected) emerge as the most robust and reliable metrics for stationary connectivity estimation.

Key Results

  • Group-Level Repeatability: Metrics affected by spatial leakage (e.g., coherence, AEC) showed very high correlations (around 0.99) between dataset halves. After leakage correction, these correlations dropped, but AEC (corrected) and partial correlation (corrected) maintained high repeatability (around 0.95). Phase-based metrics like PLI and wPLI showed lower repeatability (around 0.90 or less).
  • Within-Subject Consistency: Similar to group-level results, leakage-prone metrics showed high within-subject consistency. However, leakage correction reduced this consistency. AEC (corrected) and partial correlation (corrected) showed moderate, albeit broad, within-subject repeatability (median around 0.4-0.5). Phase-based metrics generally showed poor within-subject repeatability (median around 0.3).
  • Between-Subject Consistency: Leakage-prone metrics again showed high consistency between subjects. Among the leakage-corrected or immune metrics, AEC (corrected) and partial correlation (corrected) showed the highest between-subject correlations (median around 0.4 and 0.3, respectively).
  • PDC: Among directed measures, Partial Directed Coherence (PDC) performed best in terms of reliability, though concerns about its robustness to leakage remain.

Limitations and Future Work

  • Gaussianity Assumption: The spatial leakage correction method used is optimal for Gaussian-distributed data. If this assumption is violated, the correction's effectiveness might be reduced.
  • Parcellation Choice: The study used an fMRI-derived parcellation. It's suggested that a MEG-specific parcellation might yield different results for some metrics.
  • Source Reconstruction Variability: Potential variability in source reconstruction methods (e.g., beamformer orientation estimation) across runs could influence results, though this is not extensively explored.
  • Scan Length: The authors suggest that longer resting-state scans (e.g., 15 minutes or more) might be necessary for phase-based metrics to achieve better signal-to-noise ratio and thus improved reliability.
  • Task vs. Resting-State: While the findings are based on resting-state data, the authors speculate that inherent repeatability in task-based paradigms might make some less reliable resting-state metrics more efficacious when averaged over trials.
  • Future Directions: The paper strongly advocates for the use of leakage-corrected amplitude envelope correlation and partial correlation for stationary connectivity analysis. Future work could explore how these findings translate to dynamic connectivity measures or different types of source reconstruction techniques.

Summary updated

BibTeX
@article{colclough2016how,
  author = {Giles L. Colclough and Mark W. Woolrich and Prejaas Tewarie and Matthew J. Brookes and Andrew J. Quinn and Stephen M. Smith},
  title = {{How reliable are MEG resting-state connectivity metrics?}},
  journal = {NeuroImage},
  year = {2016},
  doi = {10.1016/j.neuroimage.2016.05.070},
}
Figure