Abstract
Mining genuine mechanisms underlying the complex data generation process in real-world systems is a fundamental step in promoting interpretability of, and thus trust in, data-driven models. Therefore, we propose a variation-based cause effect identification (VCEI) framework for causal discovery in bivariate systems from a single observational setting. Our framework relies on the principle of independence of cause and mechanism (ICM) under the assumption of an existing acyclic causal link, and offers a practical realization of this principle. Principally, we artificially construct two settings in which the marginal distributions of one covariate, claimed to be the cause, are guaranteed to have non-negligible variations. This is achieved by re-weighting samples of the marginal so that the resultant distribution is notably distinct from this marginal according to some discrepancy measure. In the causal direction, such variations are expected to have no impact on the effect generation mechanism. Therefore, quantifying the impact of these variations on the conditionals reveals the genuine causal direction. Moreover, we formulate our approach in the kernel-based maximum mean discrepancy, lifting all constraints on the data types of cause-and-effect covariates, and rendering such artificial interventions a convex optimization problem. We provide a series of experiments on real and synthetic data showing that VCEI is, in principle, competitive to other cause effect identification frameworks.
Introduction
Distinguishing between correlation and causation remains a central challenge in data science, particularly when restricted to a single observational dataset. In bivariate systems—where only two variables, $X$ and $Y$, are observed—traditional methods like conditional independence tests are inapplicable because there are no additional variables to condition upon.
Existing approaches often rely on restrictive assumptions, such as additive noise models or specific functional forms (e.g., linear or Gaussian). Furthermore, many current methods suffer from high computational complexity or extreme sensitivity to the choice of the underlying machine learning model. The Variation-based Cause Effect Identification (VCEI) framework addresses these limitations by providing a data-agnostic approach that leverages the principle of Independence of Cause and Mechanism (ICM) to identify causal directions without imposing rigid constraints on data types or functional relationships.
Methodology
The VCEI framework operationalizes the ICM principle, which states that if $X \to Y$, the process generating the cause $P(X)$ is independent of the mechanism $P(Y|X)$ that maps the cause to the effect. The framework identifies the causal direction by "stress-testing" the stability of these mechanisms through artificial variations.
1. Artificially Generated Variations
For a candidate cause $X$, the algorithm identifies a set of weights ($\alpha$) to apply to the existing observational data points. This re-weighting creates a "varied" distribution $P'(X)$ that is maximized for discrepancy against the original $P(X)$. This is formulated as a Maximum Mean Discrepancy (MMD) maximization problem. To ensure computational feasibility, the authors relax this into a Semidefinite Program (SDP), allowing it to be solved via convex optimization.
2. Quantifying Mechanism Impact
Once the variations are generated, the framework trains predictive models on both the original and the re-weighted data. If $X$ is the true cause, the mechanism $P(Y|X)$ should remain stable; consequently, models trained on different variations of $X$ should yield similar predictions. Conversely, if $X$ is the effect, the "acausal" mechanism $P(X|Y)$ will be highly sensitive to these artificial shifts, resulting in a high disagreement score between models.
By comparing these disagreement scores in both directions ($X \to Y$ vs. $Y \to X$), the direction exhibiting the lower score (greater stability) is identified as the true causal path. A "trend" criterion is also utilized: as the strength of the artificial variation increases, the disagreement in the acausal direction typically grows significantly, while the causal direction remains relatively flat.
Experimental Evaluation
The authors evaluated VCEI against state-of-the-art baselines using a mix of synthetic and real-world benchmarks:
- Synthetic Benchmarks: Testing included the "SIM" suite (covering confounder bias, low noise, and Gaussian scenarios) as well as Additive Noise (AN), Linear-Sigmoid (LS), and Multiplicative Noise (MN-U) datasets. VCEI achieved identification accuracies near or above 80% on many of these sets, often outperforming traditional regression-based methods.
- Real-World Data: The framework was tested on the Tübingen Cause-Effect (CE) benchmark, which contains over 100 data pairs from diverse fields like meteorology and economics. VCEI performed on par with established state-of-the-art methods, demonstrating its robustness on noisy, real-world observations.
Discussion and Conclusion
VCEI introduces a novel way to simulate interventions through convex optimization. By using kernel-based MMD, the framework is "data-type blind," making it applicable to scalars, vectors, or complex structures as long as a valid kernel exists.
However, some limitations remain. The Semidefinite Relaxation (SDR) used for optimization is computationally intensive ($O(N^2)$), necessitating the use of representative "coresets" for larger datasets. Additionally, like all ICM-based methods, VCEI may struggle with perfectly symmetric systems (e.g., linear systems with Gaussian noise) where the causal direction is mathematically indistinguishable from observational data alone. Future work will focus on addressing "disagreement bias" in high-capacity models like neural networks and improving the scalability of the optimization process.
Summary updated