Abstract
Causal discovery uncovers complex relationships between variables, enhancing predictions, decision-making, and insights into real-world systems, especially in nonlinear multivariate time series. However, most existing methods primarily focus on pairwise cause-effect relationships, overlooking interactions among groups of variables, i.e., subsystems and their collective causal influence. In this study, we introduce gCDMI, a novel multi-group causal discovery method that leverages group-level interventions on trained deep neural networks and employs model invariance testing to infer causal relationships. Our approach involves three key steps: deep learning for structural modeling, group-wise interventions via knockoffs, and model invariance testing. We evaluate our method on simulated and real-world datasets, demonstrating its superior performance in identifying group-level causal relationships compared to existing methods.
Introduction
Traditional causal discovery methods typically focus on pairwise relationships, attempting to determine if a single variable $A$ causes variable $B$. However, complex real-world systems—such as the human brain or global climate patterns—are composed of subsystems where groups of variables exert collective influence. Analyzing these systems at the individual variable level often results in "spaghetti" graphs that are difficult to interpret or contain spurious connections. Furthermore, many existing tools struggle with the high-dimensional, nonlinear interactions inherent in these systems. There is a critical need for methods that can reliably infer causality at the group level while leveraging the representational power of deep neural networks.
Methodology: The gCDMI Framework
We propose gCDMI (group Causal Discovery via Model Invariance), a framework designed to identify causal links between entire subsystems. The process follows three main stages:
- Structure Learning: A deep autoregressive recurrent neural network (DeepAR) is used to model the temporal dependencies of all variables across all groups. This allows the model to capture complex, nonlinear rules governing the system's dynamics.
- Group-wise Interventions via Knockoffs: To test if Group $X_i$ causes Group $X_j$, we perform a counterfactual intervention. Instead of simple deletion—which can push data "out-of-distribution" for neural networks—we replace $X_i$ with Knockoffs. These synthetic variables preserve the original data's statistical structure (covariance) but are decorrelated from the target, effectively "silencing" the causal signal without breaking the model's internal logic.
- Model Invariance Testing: We compare the model's prediction errors (residuals) on the original data versus the knockoff-intervened data using a Kolmogorov-Smirnov (KS) test. If the error distribution changes significantly, the group is identified as a cause. If the model remains "invariant," no causal link is inferred.
To handle high-dimensional groups, we integrate Canonical Correlation Analysis (CCA) to reduce dimensionality while preserving essential correlation structures.
Experimental Evaluation
The performance of gCDMI was evaluated using both synthetic and high-stakes real-world data:
- Synthetic SCMs: Generated using Structural Causal Models with varying levels of nonlinearity and interaction density.
- Fluxnet2015 (Climate-Ecosystem): Data from global sites measuring the relationship between climate variables (Temperature, Radiation) and ecosystem responses (Photosynthesis, Respiration).
- Moxa Geodynamic Observatory (Tectonic-Climate): Highly sensitive laser strainmeter data used to see how meteorological factors influence movements in the Earth's crust.
- NetSim (Neuroscience): A benchmark dataset of simulated fMRI time series used to map connections between different brain regions.
Results and Discussion
Our results demonstrate that gCDMI consistently outperforms baseline methods, particularly in complex, nonlinear scenarios.
Superior Accuracy and Robustness
In synthetic benchmarks, gCDMI achieved higher F-scores than baselines such as MC-CDMI, MC-PCMCI, and MC-VGC. Notably, its performance remained robust even as the interaction density within the systems increased, maintaining a competitive edge where other methods often faltered.
Insights into Real-World Systems
- Climate Science: On Fluxnet data, gCDMI was uniquely successful in identifying bidirectional causal links ($G_{Climate} \leftrightarrow G_{Ecosystem}$), accurately reflecting the feedback loops known to exist in forest ecosystems.
- Neuroscience: In fMRI simulations, gCDMI achieved 56% true positives, significantly outperforming 2GVecCI and Vanilla-PC, which struggled to distinguish cause from effect in noisy neural signals.
The reliance on deep networks allows gCDMI to maintain high accuracy where linear methods (like Granger Causality) fail due to the "curvy" or complex nature of the relationships.
Conclusion and Future Work
gCDMI represents a significant advancement in group-level causal discovery by combining deep learning with robust invariance testing. While the computational cost of training deep models for every subsystem is higher than linear approaches, the gain in accuracy and the ability to handle nonlinear feedback loops are substantial.
Future research will focus on improving the scalability of gCDMI for massive systems, such as whole-brain models. Additionally, we aim to explore the integration of Large Language Models (LLMs) to incorporate domain-specific knowledge as a "common sense" check on the discovered causal structures.
Summary updated