Causal Discovery with General Non-Linear Relationships Using Non-Linear ICA

Ricardo Pio Monti, Kun Zhang, Aapo Hyvärinen

TL;DR

This paper introduces NonSENS, a novel framework for causal discovery that can handle general non-linear relationships between variables, a significant limitation of previous methods. By leveraging non-linear Independent Component Analysis (ICA) and exploiting non-stationarity in data, NonSENS can infer causal directions even when relationships are complex and not simply additive.

What Problem Does This Paper Address?

Understanding cause-and-effect relationships is crucial in science, but obtaining this information from observational data (where we can't directly intervene) is challenging. Many existing causal discovery methods rely on strong assumptions, such as linear relationships between variables or additive noise. These assumptions are often violated in real-world scenarios, limiting the applicability of these methods. For instance, biological systems, financial markets, or neural networks often exhibit complex, non-linear interactions that cannot be captured by linear models. This paper aims to bridge this gap by developing a method that can discover causal structures in the presence of general non-linear relationships, without requiring the assumption of additive noise.

Methods

The core of the proposed method, NonSENS, is built upon recent advancements in non-linear Independent Component Analysis (ICA), specifically the Time Contrastive Learning (TCL) algorithm. The key idea is to exploit non-stationarity in the data. If data comes from different experimental conditions or time periods, the underlying latent sources (the true drivers of the observed variables) will exhibit changes in their statistical properties across these conditions. TCL learns a feature extractor that can distinguish between these different conditions, effectively disentangling the latent sources.

NonSENS then uses this disentangled information in two main ways:

  1. Independence Tests: By treating the disentangled latent sources as potential causal drivers, the method performs a series of independence tests. If a variable is causally influenced by another, its latent disturbance will be independent of the other variable's latent disturbance.
  2. Likelihood Ratio Measure: An alternative approach proposes a novel measure of causal direction based on the likelihood ratio. This measure is derived from asymptotic approximations and is designed to be more statistically efficient than independence tests.

The method also includes a specialized linear ICA algorithm to handle situations with a limited number of experimental conditions, which can be a challenge for standard TCL. For multivariate causal discovery, NonSENS is combined with a constraint-based method like the PC algorithm.

Datasets

The paper uses two types of datasets for evaluation:

  • Synthetic Data: Generated using a non-linear ICA model with varying degrees of non-linearity, number of experimental conditions, and observations per condition. This allows for controlled experiments to assess the method's performance under different scenarios.
  • Hippocampal fMRI Data: Real-world data from the MyConnectome project, consisting of resting-state fMRI scans from a single subject over 84 days. Each day is treated as a distinct experimental condition, and data from six brain regions are analyzed. This application demonstrates the method's ability to handle complex, real-world biological data.

Key Insights and Ideas

  • Non-stationarity as a Causal Signal: The paper highlights that changes in data distribution over time or across experimental conditions (non-stationarity) are not just noise but can be a powerful signal for inferring causal relationships, especially in non-linear settings.
  • TCL as a Foundation for Causal Discovery: By adapting TCL, which is primarily for source separation, the authors show its utility in a causal inference context. The ability of TCL to disentangle non-stationary latent sources is key.
  • General Non-linearities are Tractable: The most significant insight is that causal discovery can be performed even with complex, arbitrary non-linear relationships, moving beyond the restrictive additive noise assumption common in prior work.
  • Bridging ICA and SEMs: The work establishes a formal connection between non-linear ICA models and structural equation models (SEMs), providing a theoretical basis for using ICA techniques for causal discovery.

Key Results

  • Bivariate Causal Discovery: NonSENS consistently outperforms existing methods (including linear ICA-based methods, RESIT, and CD-NOD) in identifying the correct causal direction, especially as the non-linearity of the causal relationships increases. This is demonstrated across various depths of the neural network used for feature extraction.
  • Multivariate Causal Discovery: In multivariate settings, NonSENS, when combined with the PC algorithm, shows competitive performance, particularly in resolving undirected edges, leading to better estimated DAGs compared to the PC algorithm alone.
  • Real-world Application: Applied to fMRI data, NonSENS successfully recovers feasible directed connectivity structures within the hippocampus, aligning with known anatomical connectivity.

Limitations and Future Work

  • Computational Efficiency: The paper notes that extending NonSENS to multivariate cases is not straightforward and suggests combining it with existing constraint-based methods. Developing more computationally efficient multivariate extensions is a key area for future work.
  • Assumptions: While NonSENS relaxes many assumptions, it still relies on the presence of non-stationarity and the absence of hidden confounders. The identifiability of causal direction is guaranteed under these conditions.
  • Data Requirements: The method's performance is shown to improve with more experimental conditions and observations per condition, suggesting that sufficient non-stationarity is crucial.

The authors explicitly state that developing computationally more efficient methods for the multivariate case is a line of future work.

Summary updated

Abstract

We consider the problem of inferring causal relationships between two or more passively observed variables. While the problem of such causal discovery has been extensively studied especially in the bivariate setting, the majority of current methods assume a linear causal relationship, and the few methods which consider non-linear dependencies usually make the assumption of additive noise. Here, we propose a framework through which we can perform causal discovery in the presence of general non-linear relationships. The proposed method is based on recent progress in non-linear independent component analysis and exploits the non-stationarity of observations in order to recover the underlying sources or latent disturbances. We show rigorously that in the case of bivariate causal discovery, such non-linear ICA can be used to infer the causal direction via a series of independence tests. We further propose an alternative measure of causal direction based on asymptotic approximations to the likelihood ratio, as well as an extension to multivariate causal discovery. We demonstrate the capabilities of the proposed method via a series of simulation studies and conclude with an application to neuroimaging data.
BibTeX
@inproceedings{monti2019causal,
  author = {Ricardo Pio Monti and Kun Zhang and Aapo Hyvärinen},
  title = {{Causal Discovery with General Non-Linear Relationships Using Non-Linear ICA}},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  year = {2019},
  url = {https://openreview.net/forum?id=H6PEn8SFixq},
}
Figure