Abstract
Existing approaches to causal discovery often rely on restrictive modeling assumptions, such as additive noise, which limit their applicability in real-world settings involving heavy-tailed data or mixed variable types. We introduce Conditionally Parametric Causal Models (CPCM), a flexible framework where the conditional distribution of an effect given its cause belongs to a known parametric family. We establish the identifiability of CPCMs using sufficient statistics and propose a robust algorithm for structure estimation. Our methodology demonstrates state-of-the-art performance, particularly in non-additive and heteroscedastic scenarios.
Introduction
Determining the direction of causality from observational data is a fundamental challenge in science. Most existing methods rely on Additive Noise Models (ANM), which assume that an effect $Y$ is a function of its cause $X$ plus an independent noise term $\epsilon$: $Y = f(X) + \epsilon$. While mathematically convenient, ANMs assume the cause only influences the mean of the effect's distribution. In reality, causes often influence higher-order moments, such as variance (heteroscedasticity) or tail behavior. Furthermore, ANMs struggle with mixed data types, such as continuous causes producing discrete, count-based outcomes.
The CPCM Framework
To address these limitations, we propose Conditionally Parametric Causal Models (CPCM). In this framework, the conditional distribution $P(Y|X)$ is assumed to belong to a known parametric family (e.g., Gaussian, Poisson, Gamma, or Pareto). This allows the cause to influence any parameter of the effect's distribution, rather than just shifting the mean.
The core of our approach lies in the use of sufficient statistics. We prove that if the conditional distribution belongs to the exponential family, the causal direction is identifiable unless the relationship between the parameters and the data follows a specific, rare linear form. This "distributional fingerprint" ensures that the true causal model is distinguishable from its reverse.
Methodology
We developed a score-based algorithm for estimating causal structures:
- Parameter Estimation: For a candidate direction $X \to Y$, we estimate the parameters of the chosen distribution family.
- Residual Analysis: We calculate generalized residuals using a probability integral transform to map the data to a Uniform distribution.
- Independence Testing: We test if these residuals are independent of the cause.
- Multivariate Scaling: For larger graphs, we use RESIT-greedy, a hybrid algorithm that identifies a topological ordering before pruning edges to minimize the Structural Intervention Distance (SID).
To prevent overfitting, we employ Sequential Family Selection, starting with simple one-parameter models and increasing complexity only when necessary.
Experimental Results
Our evaluation across synthetic and real-world datasets highlights the robustness of CPCM:
- Non-Additive Performance: In scenarios involving Poisson (POISg) and Pareto (PARg) distributions, CPCM achieved 90-94% accuracy in identifying causal direction. In contrast, standard ANM-based methods dropped to 0-12% accuracy, as they could not account for the non-additive nature of the data.
- Multivariate Graphs: In 5-node simulations, CPCM consistently outperformed baselines like LiNGAM and the PC algorithm, achieving significantly lower SID scores.
- Real-World Application: We applied CPCM to the French Motor Third-Party Liability insurance dataset. CPCM successfully identified plausible relationships, such as Vehicle Age $\to$ Claim Number, whereas linear models produced counterintuitive results (e.g., Policy Duration $\to$ Vehicle Age).
Conclusion
CPCM provides a powerful alternative to additive noise models by leveraging the flexibility of parametric families. By focusing on the entire conditional distribution rather than just the mean, CPCM enables reliable causal discovery in complex, real-world datasets characterized by heteroscedasticity and mixed data types. Future work will explore extensions to time-series data and methods for handling unobserved confounders.
Summary updated