Abstract
We consider the problem of inferring causal relationships between two or more passively observed variables. While the problem of such causal discovery has been extensively studied, especially in the linear case, identifying causal direction in general non-linear, non-additive noise settings remains a significant challenge. We propose NonSENS, a framework that leverages non-linear Independent Component Analysis (ICA) and the non-stationarity of data to identify underlying causal structures without requiring restrictive functional assumptions like linearity or additive noise.
Introduction
In data science, the distinction between correlation and causation is fundamental. While randomized controlled trials are the gold standard for causal inference, they are often impractical or unethical to perform. Causal discovery aims to infer these relationships from passive observational data. However, most existing methods rely on strict assumptions, such as linear relationships or Additive Noise Models (ANMs). When real-world systems exhibit complex non-linearities and non-additive noise, these traditional tools often fail to identify the correct causal direction. This paper introduces a method to tackle the general case where relationships can be arbitrarily complex.
The NonSENS Framework
The authors introduce NonSENS (Non-linear SEM Estimation using Non-Stationarity) to address these limitations. The framework is built upon the realization that non-stationarity—changes in data across different regimes or time segments—is a powerful feature for identifiability rather than a nuisance to be removed.
Structural Equation Modeling
NonSENS models causal relationships using general non-linear structural equation models (SEMs). Unlike ANMs, these models allow for arbitrary interactions between observed variables and latent noise terms, providing a more flexible representation of real-world phenomena.
Methodology
The core of the method involves three main steps:
- Non-linear ICA via Time Contrastive Learning (TCL): By assuming data is collected across different "regimes," a neural network is trained to classify which segment a data point belongs to. The hidden layers of this network are mathematically proven to contain a linear mixture of the true independent causal sources (noise variables).
- Recovering the Noise: Once hidden representations are obtained, a novel linear ICA algorithm based on score matching is used to fully unmix the signals and recover the latent disturbances ($N_1, N_2, \dots$).
- Determining Direction: Causal direction is then determined using either independence testing (via the Hilbert-Schmidt Independence Criterion) or a more statistically efficient Likelihood Ratio measure.
Experimental Evaluation
The performance of NonSENS was evaluated using both synthetic and real-world datasets, demonstrating its robustness in complex settings.
Synthetic Data Results
Synthetic datasets were generated using "mixing-DNNs" with depths ranging from 1 to 5 layers to simulate increasing levels of non-linearity. The results show that while standard methods like LiNGAM perform well in linear settings (1 layer), their accuracy drops significantly as complexity increases. In contrast, NonSENS remains robust, maintaining high accuracy even in 5-layer DNN settings.
Real-world Validation: Hippocampal fMRI
The method was applied to resting-state fMRI data from the MyConnectome project, consisting of scans over 84 successive days. NonSENS successfully identified directed edges between brain regions, such as the Entorhinal Cortex to the Subiculum, which align with known anatomical connectivity in the hippocampus.
Conclusion
NonSENS represents a significant advancement in causal discovery by relaxing the restrictive assumptions of linearity and additive noise. By exploiting non-stationarity through non-linear ICA, it provides a robust framework for identifying causal directions in complex systems. While the method currently requires data from multiple regimes and faces scaling challenges in high-dimensional multivariate cases, it sets a new benchmark for general non-linear causal inference.
Summary updated