SELF: Structural Equational Likelihood Framework for Causal Discovery

Ruichu Cai, Jie Qiao, Zhenjie Zhang, Zhifeng Hao

Abstract

Causal discovery from observational data is hindered by Markov equivalence classes and the scalability-accuracy trade-off. The Structural Equational Likelihood Framework (SELF) addresses these by unifying global Bayesian network scoring with local structural equation modeling. By maximizing the likelihood of noise residuals estimated via XGBoost, SELF effectively identifies causal directions in complex linear and nonlinear systems.

Introduction

Determining "what causes what" from purely observational data is a fundamental challenge in science. Traditional methods often struggle with Markov Equivalence Classes (MECs), where multiple causal graphs are statistically indistinguishable (e.g., $X \to Y$ vs. $Y \to X$). While "local" methods like Additive Noise Models (ANM) can break these ties by analyzing noise complexity, they often fail to scale to large networks. SELF bridges this gap, offering a framework that is globally consistent across a network while remaining locally sensitive enough to resolve causal directions accurately.

Methods

The core innovation of SELF is transforming the standard log-likelihood of the observed data into a log-likelihood of the noise residuals.

Likelihood transformation in SELF

Structural Equational Model (SEM)

SELF assumes an additive noise model where each variable $X_i$ is a function of its parents plus independent noise: $X_i = F_i(Parents_i) + E_i$. The framework proves that the probability of observing a variable is equivalent to the probability of its noise residual. Consequently, the optimal causal graph is the one that maximizes the likelihood of these estimated noises across the entire network.

XGBoost Integration

To estimate the functions $F_i$, SELF utilizes XGBoost. It employs GBLinear for linear relationships and GBTree for complex nonlinear ones, allowing it to handle diverse data types that traditional linear-only methods (like LiNGAM) cannot.

Search and Regularization

The framework uses a hill-climbing search to navigate the graph space. To prevent overfitting, it incorporates a Bayesian Information Criterion (BIC) penalty, favoring sparser, more efficient graphs.

Datasets and Evaluation

SELF was evaluated against state-of-the-art baselines (LiNGAM, DLiNGAM, HCBN, and MMPC-ANM) using:

  • Synthetic Data: Randomly generated structures with up to 8,000 samples and 60 variables, covering both linear non-Gaussian and nonlinear data.
  • Real-World Benchmarks: Classic Bayesian networks from the bnlearn repository, including Child (20 nodes), Alarm (37 nodes), Win95pts (76 nodes), and Pathfinder (135 nodes).

Key Results

SELF demonstrates significant improvements in both accuracy and the ability to resolve Markov equivalence.

Performance of SELF on linear and nonlinear data
  • Breaking Equivalence: In tests targeting Markov equivalence edges, SELF remained robust while traditional likelihood methods like HCBN saw accuracy plummet.
  • Superior Accuracy: On linear data, SELF consistently outperformed LiNGAM and DLiNGAM. On nonlinear data, it significantly surpassed MMPC-ANM.
  • Scalability: SELF maintained high precision and recall even on the 135-node Pathfinder network, proving its capability for large-scale systems.
  • Precision: On real-world datasets, SELF showed particularly high precision, effectively avoiding the "false discovery" of causal links.

Discussion and Future Work

SELF treats noise as a "causal fingerprint." In the correct causal direction, noise should be independent of the cause, a property SELF formalizes by showing that the true structure minimizes noise entropy. By leveraging modern machine learning tools like XGBoost, SELF demonstrates that causal discovery can benefit from high-performance gradient boosting.

However, SELF relies on the Additive Noise Model (ANM) assumption. Performance may degrade if the data generation process involves multiplicative noise. Future work includes extending SELF to support other causal mechanisms and exploring parallelization to handle even larger datasets.

Summary updated

Abstract

Causal discovery without intervention is well recognized as a challenging yet powerful data analysis tool, boosting the development of other scientific areas, such as biology, astronomy, and social science. The major technical difficulty behind the observation-based causal discovery is to effectively and efficiently identify causes and effects from correlated variables given the existence of significant noises. Previous studies mostly employ two very different methodologies under Bayesian network framework, namely global likelihood maximization and locally complexity analysis over marginal distributions. While these approaches are effective in their respective problem domains, in this paper, we show that they can be combined to formulate a new global optimization model with local statistical significance, called structural equational likelihood framework (or SELF in short). We provide thorough analysis on the soundness of the model under mild conditions and present efficient heuristic-based algorithms for scalable model training. Empirical evaluations using XGBoost validate the superiority of our proposal over state-of-the-art solutions, on both synthetic and real world causal structures.
BibTeX
@inproceedings{cai2018self,
  author = {Ruichu Cai and Jie Qiao and Zhenjie Zhang and Zhifeng Hao},
  title = {{SELF: Structural Equational Likelihood Framework for Causal Discovery}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2018},
  doi = {10.1609/aaai.v32i1.11566},
}
Figure