TL;DR
This paper introduces "Population Post Selection Inference" (popPSI), a new statistical challenge in analyzing brain connectivity across groups. They propose a novel method, R³, that significantly improves accuracy and power compared to existing approaches by accounting for two sources of variability in brain network estimation.
What Problem Does This Paper Address?
Neuroscientists often use functional Magnetic Resonance Imaging (fMRI) to study brain connectivity, essentially mapping how different brain regions interact. This is typically done by building "brain networks" where regions are nodes and connections are edges. A key question is how these networks differ between groups, for example, between healthy individuals and those with a neurological condition like autism.
The standard approach involves:
- Estimating a brain network for each individual subject.
- Assuming these individual networks are fixed and known.
- Performing statistical tests on specific connections (edges) across the groups.
However, this standard method has a major flaw: it doesn't account for the fact that individual brain networks are estimated, not perfectly known. This estimation process introduces variability, leading to an underestimation of true variability and consequently, a high rate of false positives (claiming a connection differs when it doesn't) and low statistical power (failing to detect real differences). This paper tackles this "Population Post Selection Inference" (popPSI) problem, aiming to develop a method that correctly handles this estimation variability.
Methods
The authors propose a novel method called R³ (Resampling, Random Penalization, and Random Effects) to address the popPSI problem. The core idea is to model the problem at two levels:
- Subject-Level: Each subject has their own brain network, estimated from their fMRI data.
- Group-Level: The probability of a specific connection existing is governed by a group-level parameter.
R³ integrates three key components:
- Resampling (Bootstrap): To estimate the variability introduced by estimating each subject's network, they use bootstrap resampling. This generates multiple "replicate" networks for each subject, allowing for a better estimation of network variability.
- Random Penalization: To mitigate biases caused by graph selection errors (where the estimation process incorrectly identifies edges), they introduce random regularization penalties. This technique helps to stabilize the graph estimation process and reduce the impact of these errors.
- Random Effects (Beta-Binomial Model): To properly account for both subject-level estimation variability and group-level differences, they employ a two-level Beta-Binomial model. This model is specifically designed for binary data (edge present/absent) with overdispersion, which is characteristic of this problem. They then use a two-sample random effects test statistic derived from this model.
By combining these elements, R³ aims to provide more accurate and powerful inference on functional connectivity differences between groups.
Datasets
The authors apply their method to the UCLA Sample 1 from the Autism Brain Imaging Data Exchange (ABIDE) dataset. This dataset consists of resting-state fMRI data from 73 subjects (41 with Autism Spectrum Disorder and 32 controls). The data was preprocessed and parcellated into 113 regions of interest.
Key Insights and Ideas
- The "Post Selection Inference" Problem: The core challenge is that standard statistical methods assume data is directly observed, but in neuroimaging, we're inferring connections from noisy, estimated data. This "post-selection" aspect is critical and often ignored.
- Two Levels of Variability: Brain network analysis suffers from variability between subjects (different people have different brain structures) and within subjects (the estimation process itself is noisy). Ignoring either leads to flawed conclusions.
- Graph Selection Errors are Non-Random: Unlike random noise, errors in estimating brain networks (e.g., falsely identifying an edge) are not uniformly distributed. They can be systematically biased, especially in complex or correlated network structures, leading to biased group-level estimates.
- R³ as a Holistic Solution: The R³ method's strength lies in its integration of resampling, random penalization, and random effects. Each component addresses a specific challenge, and their combination leads to substantial improvements.
Key Results
- Simulation Studies: In extensive simulations, R³ consistently outperformed the standard approach and its own variants (R² with only resampling/random effects, or resampling/random penalization).
- R³ achieved significantly higher True Positive Rates (TPR) and lower False Discovery Proportions (FDP) across various network structures (small-world, banded, hub) and scenarios (clustered vs. random differential edges).
- For example, in one scenario, R³ achieved a TPR of 0.96 with an FDP of 0.15, while the standard method had a TPR of 0.25 with an FDP of 0.74.
- The method showed robustness to variations in differential edge probability and network density.
- Autism Case Study: Applying R³ to the ABIDE dataset identified 7 differential functional connections between autistic subjects and controls. These findings align with existing literature on ASD, suggesting increased short-range connections within hemispheres and decreased long-range connections between hemispheres in ASD subjects.
Limitations and Future Work
- FDR Control: The authors acknowledge that their method does not provably control the False Discovery Rate (FDR) at 10% in all simulated scenarios, particularly those with severe violations of irrepresentable conditions (which are common in real fMRI data). They suggest that achieving strict FDR control might be impossible under such challenging conditions.
- Theoretical Challenges: The paper focuses on methodological solutions and building intuition. Many open theoretical questions remain, including establishing provable FDR control and understanding the relationship between subject-level graph selection error rates and group-level inference error rates.
- Generalizability: While R³ is designed for Gaussian Graphical Models, extensions to other network models (e.g., sparse covariance models) and other types of inference (e.g., multiple groups, continuous outcomes) are suggested.
- Data Preprocessing: The paper notes that the choice of brain parcellation method and handling of physiological artifacts (like head motion) are important considerations for fMRI data that warrant further investigation.
- Software Availability: The authors mention that software for their R³ procedure will be available as part of the Markov Network Matlab Toolbox.
Summary updated