Identifiability of causal graphs under nonadditive conditionally parametric causal models

Juraj Bodik, Valérie Chavez‐DemoulinarXiv 2023 · DOI · arXiv

Abstract

Existing approaches to causal discovery often rely on restrictive modeling assumptions, such as additive noise, which limit their applicability in real-world settings involving heavy-tailed data or mixed variable types. We introduce Conditionally Parametric Causal Models (CPCM), a flexible framework where the conditional distribution of an effect given its cause belongs to a known parametric family. We establish the identifiability of CPCMs using sufficient statistics and propose a robust algorithm for structure estimation. Our methodology demonstrates state-of-the-art performance, particularly in non-additive and heteroscedastic scenarios.

Introduction

Determining the direction of causality from observational data is a fundamental challenge in science. Most existing methods rely on Additive Noise Models (ANM), which assume that an effect $Y$ is a function of its cause $X$ plus an independent noise term $\epsilon$: $Y = f(X) + \epsilon$. While mathematically convenient, ANMs assume the cause only influences the mean of the effect's distribution. In reality, causes often influence higher-order moments, such as variance (heteroscedasticity) or tail behavior. Furthermore, ANMs struggle with mixed data types, such as continuous causes producing discrete, count-based outcomes.

The CPCM Framework

To address these limitations, we propose Conditionally Parametric Causal Models (CPCM). In this framework, the conditional distribution $P(Y|X)$ is assumed to belong to a known parametric family (e.g., Gaussian, Poisson, Gamma, or Pareto). This allows the cause to influence any parameter of the effect's distribution, rather than just shifting the mean.

Visualizing non-additive causal relationships: A continuous cause (X1) influencing a discrete, count-based effect (X2).

The core of our approach lies in the use of sufficient statistics. We prove that if the conditional distribution belongs to the exponential family, the causal direction is identifiable unless the relationship between the parameters and the data follows a specific, rare linear form. This "distributional fingerprint" ensures that the true causal model is distinguishable from its reverse.

Methodology

We developed a score-based algorithm for estimating causal structures:

  1. Parameter Estimation: For a candidate direction $X \to Y$, we estimate the parameters of the chosen distribution family.
  2. Residual Analysis: We calculate generalized residuals using a probability integral transform to map the data to a Uniform distribution.
  3. Independence Testing: We test if these residuals are independent of the cause.
  4. Multivariate Scaling: For larger graphs, we use RESIT-greedy, a hybrid algorithm that identifies a topological ordering before pruning edges to minimize the Structural Intervention Distance (SID).

To prevent overfitting, we employ Sequential Family Selection, starting with simple one-parameter models and increasing complexity only when necessary.

Experimental Results

Our evaluation across synthetic and real-world datasets highlights the robustness of CPCM:

  • Non-Additive Performance: In scenarios involving Poisson (POISg) and Pareto (PARg) distributions, CPCM achieved 90-94% accuracy in identifying causal direction. In contrast, standard ANM-based methods dropped to 0-12% accuracy, as they could not account for the non-additive nature of the data.
  • Multivariate Graphs: In 5-node simulations, CPCM consistently outperformed baselines like LiNGAM and the PC algorithm, achieving significantly lower SID scores.
  • Real-World Application: We applied CPCM to the French Motor Third-Party Liability insurance dataset. CPCM successfully identified plausible relationships, such as Vehicle Age $\to$ Claim Number, whereas linear models produced counterintuitive results (e.g., Policy Duration $\to$ Vehicle Age).

Conclusion

CPCM provides a powerful alternative to additive noise models by leveraging the flexibility of parametric families. By focusing on the entire conditional distribution rather than just the mean, CPCM enables reliable causal discovery in complex, real-world datasets characterized by heteroscedasticity and mixed data types. Future work will explore extensions to time-series data and methods for handling unobserved confounders.

Summary updated

Abstract

Existing approaches to causal discovery often rely on restrictive modeling assumptions that limit their applicability in real-world settings, particularly when data are heavy-tailed or contain a mixture of discrete and continuous variables. Identifiability of causal graphs has been established under several structural models, including linear non-Gaussian models, post-nonlinear models, and location-scale models. However, these frameworks may not capture the diversity of distributions observed in practice. To address this, we introduce Conditionally Parametric Causal Models (CPCM), a flexible class of models where the conditional distribution of the effect, given its cause, belongs to a known parametric family such as Gaussian, Poisson, Gamma, or Pareto. These models are adaptable to a wide range of practical situations, where the cause influences not only the mean but also the variance or tail behavior of the effect. We demonstrate the identifiability of CPCM by leveraging the concept of sufficient statistics. Furthermore, we propose an algorithm for estimating the causal structure from random samples drawn from CPCM. We evaluate the empirical properties of our methodology on various datasets, demonstrating state-of-the-art performance across multiple benchmarks.
BibTeX
@misc{bodik2023identifiability,
  author = {Juraj Bodik and Valérie Chavez‐Demoulin},
  title = {{Identifiability of causal graphs under nonadditive conditionally parametric causal models}},
  year = {2023},
  eprint = {2303.15376},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2303.15376},
}
Figure