Bayesian Discovery of Linear Acyclic Causal Models

Patrik O. Hoyer, Antti HyttinenUAI 2009

Bayesian Discovery of Linear Acyclic Causal Models

Introduction

Causal discovery aims to identify the directional relationships between variables from observational data. Traditionally, Linear Structural Equation Models (SEMs) have been used, but they face two significant hurdles: the "Gaussian Ambiguity"—where $X \to Y$ and $Y \to X$ are indistinguishable under Gaussian noise—and the "Uncertainty Gap"—where non-Gaussian methods like LiNGAM provide a single graph estimate without a measure of confidence. This paper introduces BayesLiNGAM, a Bayesian score-based framework that leverages non-Gaussianity for identification while providing a posterior probability to quantify model certainty.

Methodology

The core of BayesLiNGAM is the calculation of the marginal likelihood of the data given a Directed Acyclic Graph (DAG). To accommodate non-Gaussianity, the authors employ two flexible density families for noise modeling:

  • Generalized Laplace (GL): A symmetric distribution capable of representing varying degrees of kurtosis (peaky or flat).
  • Mixture of Gaussians (MoG): A highly flexible model for skewed or multi-modal distributions.

Due to the complexity of the marginal likelihood integral, a Laplace approximation is used for computational efficiency. Furthermore, the model utilizes parameter modularity, allowing the graph score to be decomposed into local node-parent scores, facilitating the evaluation of larger networks.

Experimental Evaluation

The framework was evaluated using synthetic 2-node DAGs with varying sample sizes ($N$) and non-Gaussianity parameters ($q$), as well as larger networks (3-5 nodes). Additionally, "re-simulated" real-world datasets from meteorological, biological, and economic domains were used to test performance on realistic distributions with known ground-truth causal structures.

Key Results and Insights

BayesLiNGAM demonstrates a "best of both worlds" capability. Under Gaussian conditions, it matches the performance of standard Bayesian Gaussian methods (e.g., Geiger & Heckerman), identifying the correct equivalence class. As non-Gaussianity increases, the model automatically exploits this signal to identify the exact causal direction.

Performance comparison across varying sample sizes and degrees of non-Gaussianity (q).

As shown in Figure 4, BayesLiNGAM (BL-GL and BL-MoG) significantly outperforms the original LiNGAM algorithm at small sample sizes ($N < 100$). While LiNGAM requires large datasets to converge, the Bayesian approach provides robust estimates even with limited data. It also maintains high performance across various types of data distributions (binary, class, log, quadratic).

A critical feature of the Bayesian framework is its calibration. The posterior probabilities generated by the model accurately reflect the likelihood of the predicted graph being correct.

Calibration plots showing the relationship between predicted posterior probabilities and the actual percentage of true graphs.

Figure 5 illustrates that if the model assigns an 80% probability to a specific graph, that graph is indeed correct approximately 80% of the time in simulations. This provides researchers with a reliable measure of confidence in their causal inferences, a feature missing from previous non-Gaussian point estimators.

Conclusion and Limitations

BayesLiNGAM offers a unified, robust approach to causal discovery that bridges the gap between Gaussian and non-Gaussian methods. While computational complexity remains a challenge for very large networks—necessitating greedy search strategies—the framework's ability to handle small samples and provide calibrated uncertainty estimates represents a significant advancement. Current limitations include the assumption of causal sufficiency (no hidden confounders) and linear relationships, which provide clear avenues for future research.

Summary updated

BibTeX
@inproceedings{hoyer2009bayesian,
  author = {Patrik O. Hoyer and Antti Hyttinen},
  title = {{Bayesian Discovery of Linear Acyclic Causal Models}},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  year = {2009},
}
Figure