Foundations of Causal ML

Erich Kummerfeld, Bryan Andrews, Sisi MaHealth 2024 · DOI · OpenAlex

Abstract

The present chapter covers the important dimension of causality in ML both in terms of causal structure discovery and causal inference. The vast majority of biomedical ML focuses on predictive modeling and does not address causal methods, their requirements and properties. Yet these are essential for determining and assisting patient-level or healthcare-level interventions toward improving a set of outcomes of interest. Moreover causal ML techniques can be instrumental for health science discovery.

Introduction: Beyond Predictive Black Boxes

The vast majority of machine learning applications in medicine today focus on predictive modeling. These models answer questions like, "What is the probability that this patient will relapse?" While valuable for risk stratification, such models are fundamentally limited: they cannot explain why a patient might relapse or identify which specific intervention would prevent it.

Predictive knowledge is essentially associative, capturing co-variation in data. However, clinical action requires causal knowledge—an understanding of how manipulating one factor (an intervention) changes another (the outcome). For instance, while being in a rehabilitation facility might be strongly associated with relapse risk, preventing a patient from entering rehab will not stop the relapse, as the facility is an effect of the underlying problem, not its cause. This paper provides a theoretical and practical framework to bridge this gap using Causal ML.

Theoretical Foundations of Causal ML

The paper synthesizes several core frameworks and algorithmic families necessary for causal reasoning:

  • Causal Probabilistic Graphical Models (CPGMs): These utilize Directed Acyclic Graphs (DAGs) to map "parent-child" relationships between variables.
  • Causal Markov Condition (CMC): This assumes a variable is independent of its non-descendants given its immediate causes, effectively stating that causes "shield" their effects from more distant history.
  • d-separation: A formal set of rules used to determine if a path between two variables is "blocked" or "open," which is critical for identifying confounders.
  • Structural Equation Models (SEMs): These represent functional relationships between causes and effects, allowing for the calculation of "do-calculus" to simulate interventions.

Causal structure discovery algorithms are categorized into constraint-based (e.g., PC, FCI), score-based (e.g., GES), and hybrid/functional approaches (e.g., MMHC, LiNGAM).

Case Study: The Alcohol Use Disorder (AUD) Vignette

To illustrate these concepts, the authors use a hypothetical clinical vignette involving "Amy Anonymous," a patient with Alcohol Use Disorder. The model includes variables such as Treatment Group, Adherence, Motivation, Fast Food Consumption, Stress Levels, and Days Abstained.

Causal graph for the Alcohol Use Disorder (AUD) clinical vignette showing relationships between behavioral and clinical variables.

This example highlights the "Amy Paradox": a predictive model might show that fast food consumption is associated with alcohol relapse. However, the causal model reveals that both are caused by a common factor: Stress. Consequently, an intervention targeting fast food consumption will fail to improve sobriety because the causal link is non-existent. The graph also demonstrates the danger of colliders (X → Y ← Z), where conditioning on a common effect (Y) can create false associations between independent causes (X and Z).

The 6-Step Protocol for Health Science Causal ML

A significant contribution of this work is a qualitative 6-step protocol for implementing causal ML in health sciences:

  1. Define the Goal: Determine if the objective is causal or purely predictive.
  2. Preprocess Data: Handle deterministic relationships and ensure measurement specificity.
  3. Structure Discovery: Use algorithms to find the underlying causal graph.
  4. Effect Estimation: Quantify the impact of interventions using the "do" operator.
  5. Assess Quality: Use techniques like bootstrapping to check for stability and generalizability.
  6. Implementation: Use the findings to guide actual clinical experiments or randomized controlled trials (RCTs).

Limitations and Future Directions

Despite its potential, Causal ML faces several hurdles. Discovering the correct variables to condition on is difficult when key factors are unmeasured (latent variables). Furthermore, as the number of variables increases, the computational complexity of global discovery becomes NP-hard.

Real-world Electronic Health Record (EHR) data also presents challenges, including deterministic traps (e.g., BMI being a function of height and weight) and non-random missing data. Future research must focus on adapting causal algorithms to the specific irregularities and temporal complexities of EHR data to make these models truly actionable in clinical settings.

Summary updated

Abstract

Abstract The present chapter covers the important dimension of causality in ML both in terms of causal structure discovery and causal inference. The vast majority of biomedical ML focuses on predictive modeling and does not address causal methods, their requirements and properties. Yet these are essential for determining and assisting patient-level or healthcare-level interventions toward improving a set of outcomes of interest. Moreover causal ML techniques can be instrumental for health science discovery.
BibTeX
@inproceedings{kummerfeld2024foundations,
  author = {Erich Kummerfeld and Bryan Andrews and Sisi Ma},
  title = {{Foundations of Causal ML}},
  booktitle = {Health},
  year = {2024},
  doi = {10.1007/978-3-031-39355-6\_4},
}
Figure