A Simulation Study of Three Related Causal Data Mining Algorithms

Subramani Mani, Gregory F. CooperAISTATS 2001

Abstract

This paper evaluates three variants of the Local Causal Discovery (LCD) algorithm—LCDa, LCDb, and LCDc—using simulated data to assess their ability to distinguish true causal relationships from confounded correlations. By increasing the stringency of independence tests through the requirement of multiple "trigger" variables, the authors demonstrate that the LCDc variant significantly reduces false positive rates (FPR) while maintaining comparable true positive rates (TPR).

Introduction

Determining causality is a fundamental goal in scientific research, traditionally achieved through Randomized Controlled Trials (RCTs). However, RCTs are often impractical due to ethical, logistical, or financial constraints. Causal data mining offers an alternative by inferring cause-and-effect relationships from observational data. The primary challenge in this field is distinguishing between true causation and correlation resulting from hidden common causes (confounding). This study utilizes a simulation of the ALARM medical network to benchmark the accuracy of the LCD algorithm and its variants.

Methodology

The Local Causal Discovery (LCD) algorithm operates within the framework of Causal Bayesian Networks. It identifies causal links by examining triplets of variables $(X, Y, Z)$ and testing for specific dependency and independence conditions. Specifically, if $X$ and $Y$ are dependent, $Y$ and $Z$ are dependent, and $X$ and $Z$ are dependent, but $X$ and $Z$ become independent when conditioned on $Y$, the algorithm infers that $Y$ causally influences $Z$.

Causal structures and the role of hidden variables (H) in variable triplets.

The authors compare three algorithmic variants:

  • LCDa: Requires the independence test to pass for at least one $X$ variable.
  • LCDb: Requires the test to pass for at least two different $X$ variables.
  • LCDc: Requires the test to pass for all available $X$ variables.

The evaluation used the ALARM network, a synthetic Bayesian network representing an anesthesia monitoring system with 37 nodes and 46 arcs. Datasets ranging from 50 to 5,000 instances were generated to observe how sample size affects performance.

Results and Discussion

The study found that requiring multiple constraints serves as a powerful filter against false positives. While LCDa may be susceptible to coincidental correlations, LCDc demands consistency across all available context variables, making it much harder for confounded relationships to satisfy the criteria.

True Positive Rate (TPR) and False Positive Rate (FPR) for LCD variants across different sample sizes.

As shown in the performance metrics, LCDc achieved the lowest False Positive Rate. At a sample size of 5,000, LCDa had an FPR of 0.09, whereas LCDc dropped to 0.04. Interestingly, the True Positive Rate remained consistent across all three variants, plateauing at approximately 0.27. This low TPR is expected as the algorithms are designed to be conservative, prioritizing precision over exhaustive discovery.

Conclusion

The LCDc variant provides a more reliable method for mining causal links from observational data by significantly reducing false positives. While the approach is "incomplete" by design, its local nature makes it highly scalable for large datasets. Future research should focus on developing more sensitive independence tests and applying these algorithms to more complex, real-world causal networks.

Summary updated

BibTeX
@inproceedings{mani2001simulation,
  author = {Subramani Mani and Gregory F. Cooper},
  title = {{A Simulation Study of Three Related Causal Data Mining Algorithms}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2001},
}
Figure