TL;DR
This paper introduces NonSENS, a novel framework for causal discovery that can handle general non-linear relationships between variables, a significant limitation of previous methods. By leveraging non-linear Independent Component Analysis (ICA) and exploiting non-stationarity in data, NonSENS can infer causal directions even when relationships are complex and not simply additive.
What Problem Does This Paper Address?
Understanding cause-and-effect relationships is crucial in science, but obtaining this information from observational data (where we can't directly intervene) is challenging. Many existing causal discovery methods rely on strong assumptions, such as linear relationships between variables or additive noise. These assumptions are often violated in real-world scenarios, limiting the applicability of these methods. For instance, biological systems, financial markets, or neural networks often exhibit complex, non-linear interactions that cannot be captured by linear models. This paper aims to bridge this gap by developing a method that can discover causal structures in the presence of general non-linear relationships, without requiring the assumption of additive noise.
Methods
The core of the proposed method, NonSENS, is built upon recent advancements in non-linear Independent Component Analysis (ICA), specifically the Time Contrastive Learning (TCL) algorithm. The key idea is to exploit non-stationarity in the data. If data comes from different experimental conditions or time periods, the underlying latent sources (the true drivers of the observed variables) will exhibit changes in their statistical properties across these conditions. TCL learns a feature extractor that can distinguish between these different conditions, effectively disentangling the latent sources.
NonSENS then uses this disentangled information in two main ways:
- Independence Tests: By treating the disentangled latent sources as potential causal drivers, the method performs a series of independence tests. If a variable is causally influenced by another, its latent disturbance will be independent of the other variable's latent disturbance.
- Likelihood Ratio Measure: An alternative approach proposes a novel measure of causal direction based on the likelihood ratio. This measure is derived from asymptotic approximations and is designed to be more statistically efficient than independence tests.
The method also includes a specialized linear ICA algorithm to handle situations with a limited number of experimental conditions, which can be a challenge for standard TCL. For multivariate causal discovery, NonSENS is combined with a constraint-based method like the PC algorithm.
Datasets
The paper uses two types of datasets for evaluation:
- Synthetic Data: Generated using a non-linear ICA model with varying degrees of non-linearity, number of experimental conditions, and observations per condition. This allows for controlled experiments to assess the method's performance under different scenarios.
- Hippocampal fMRI Data: Real-world data from the MyConnectome project, consisting of resting-state fMRI scans from a single subject over 84 days. Each day is treated as a distinct experimental condition, and data from six brain regions are analyzed. This application demonstrates the method's ability to handle complex, real-world biological data.
Key Insights and Ideas
- Non-stationarity as a Causal Signal: The paper highlights that changes in data distribution over time or across experimental conditions (non-stationarity) are not just noise but can be a powerful signal for inferring causal relationships, especially in non-linear settings.
- TCL as a Foundation for Causal Discovery: By adapting TCL, which is primarily for source separation, the authors show its utility in a causal inference context. The ability of TCL to disentangle non-stationary latent sources is key.
- General Non-linearities are Tractable: The most significant insight is that causal discovery can be performed even with complex, arbitrary non-linear relationships, moving beyond the restrictive additive noise assumption common in prior work.
- Bridging ICA and SEMs: The work establishes a formal connection between non-linear ICA models and structural equation models (SEMs), providing a theoretical basis for using ICA techniques for causal discovery.
Key Results
- Bivariate Causal Discovery: NonSENS consistently outperforms existing methods (including linear ICA-based methods, RESIT, and CD-NOD) in identifying the correct causal direction, especially as the non-linearity of the causal relationships increases. This is demonstrated across various depths of the neural network used for feature extraction.
- Multivariate Causal Discovery: In multivariate settings, NonSENS, when combined with the PC algorithm, shows competitive performance, particularly in resolving undirected edges, leading to better estimated DAGs compared to the PC algorithm alone.
- Real-world Application: Applied to fMRI data, NonSENS successfully recovers feasible directed connectivity structures within the hippocampus, aligning with known anatomical connectivity.
Limitations and Future Work
- Computational Efficiency: The paper notes that extending NonSENS to multivariate cases is not straightforward and suggests combining it with existing constraint-based methods. Developing more computationally efficient multivariate extensions is a key area for future work.
- Assumptions: While NonSENS relaxes many assumptions, it still relies on the presence of non-stationarity and the absence of hidden confounders. The identifiability of causal direction is guaranteed under these conditions.
- Data Requirements: The method's performance is shown to improve with more experimental conditions and observations per condition, suggesting that sufficient non-stationarity is crucial.
The authors explicitly state that developing computationally more efficient methods for the multivariate case is a line of future work.
Summary updated