Causal Deep Learning

Jeroen Berrevoets, Krzysztof Kacprzyk, Zhaozhi Qian, Mihaela van der Schaar

Abstract

Causal Deep Learning (CDL) is a unified framework designed to bridge the gap between the flexibility of deep learning and the rigor of causal inference. By organizing models along structural, parametric, and temporal dimensions, CDL provides a "topographic map" for researchers to navigate assumptions and identify research gaps. This framework enables the construction of robust pipelines for complex real-world applications in healthcare, economics, and beyond.

Introduction

Causal inference is often considered the "holy grail" of data science, as it seeks to understand the underlying mechanisms of data generation rather than mere correlations. However, a significant "practitioner's dilemma" exists: traditional causal methods often require extensive, untestable prior knowledge (e.g., a complete causal graph), while standard deep learning excels at high-dimensional data but lacks robustness to environmental changes.

Currently, there is no standardized language to describe the degree of "causality" in a deep learning model or to safely chain models together without violating underlying assumptions. CDL addresses this by providing a systematic way to "fingerprint" models based on their assumptions and capabilities.

The Causal Deep Learning Framework

The core of CDL is a three-dimensional framework that categorizes models based on the knowledge they assume or discover.

1. The Structural Dimension

The structural dimension measures the extent of knowledge regarding the links between variables. It spans three levels:

  • Unknown (Level 1): Zero prior knowledge of the structure.
  • Plausible (Level 2): Knowledge of statistical independencies or Probabilistic Graphical Models (PGMs).
  • Causal (Level 3): Knowledge of the exact directed causal graph.
The Structural Dimension of Causal Deep Learning, ranging from unknown to causal structures.

This dimension introduces a "Rung 1.5" to Pearl’s Ladder of Causality, capturing testable statistical constraints that do not fully define a causal structure but provide more information than simple association.

2. The Parametric Dimension

The parametric dimension describes the functional form of relationships between variables:

  • Non-parametric (Level 1): Total flexibility in functional form.
  • Noise Models (Level 2): Assumptions about noise distributions (e.g., additive noise).
  • Fully Parametric (Level 3): Specific parametric forms are assumed.
  • Fully Known (Level 4): The exact physical equations governing the system are known.
The Parametric Dimension of Causal Deep Learning, ranging from non-parametric to fully known functional forms.

3. The Temporal Dimension

This dimension indicates whether a model accounts for the temporal ordering of cause and effect, which is a fundamental constraint in causal reasoning.

Knowledge Transitions and Model Chaining

A key concept in CDL is the Knowledge Transition. A model performs a transition if its output provides higher-level knowledge than its input. For example, a causal discovery algorithm that takes "Unknown" structure and outputs a "Plausible" graph performs a structural transition.

This framework allows for Safe Model Chaining. By mapping models on these axes, practitioners can ensure that the output of one model (e.g., a graph learner) satisfies the input requirements of another (e.g., a treatment effect estimator requiring a known graph).

Categorizing the Machine Learning Landscape

The CDL framework reveals distinct patterns in current research:

  • Supervised Learning: Typically occupies the "Unknown Structure / Non-parametric" region, prioritizing predictive accuracy over causal understanding.
  • Treatment Effects: Research has successfully relaxed parametric assumptions but often relies on a priori known causal structures.
  • Reinforcement Learning (RL): RL and Bandits are more adept at structural transitions because they can resolve causal ambiguity through active interventions.
  • Generative Modeling: While powerful, many causal generative models assume a fully known causal graph as an input, limiting their "wild" applicability.

Discussion and Future Directions

The CDL framework highlights several "white spaces" in current research. Notably, there is a lack of deep learning models that incorporate "Fully Known" parametric factors, such as physical laws. Furthermore, the transition from "Unknown/Non-parametric" data to a "Causal/Non-parametric" representation remains a significant challenge without interventions.

The authors urge the community to "fingerprint" their work using these dimensions. This transparency regarding assumptions is crucial for the judicious adoption of causal models in high-stakes fields like healthcare and economics, where validation remains a complex but essential task.

Summary updated

Abstract

Causality has the potential to truly transform the way we solve a large number of real-world problems. Yet, so far, its potential largely remains to be unlocked as causality often requires crucial assumptions which cannot be tested in practice. To address this challenge, we propose a new way of thinking about causality -- we call this causal deep learning. Our causal deep learning framework spans three dimensions: (1) a structural dimension, which incorporates partial yet testable causal knowledge rather than assuming either complete or no causal knowledge among the variables of interest; (2) a parametric dimension, which encompasses parametric forms that capture the type of relationships among the variables of interest; and (3) a temporal dimension, which captures exposure times or how the variables of interest interact (possibly causally) over time. Causal deep learning enables us to make progress on a variety of real-world problems by leveraging partial causal knowledge (including independencies among variables) and quantitatively characterising causal relationships among variables of interest (possibly over time). Our framework clearly identifies which assumptions are testable and which ones are not, such that the resulting solutions can be judiciously adopted in practice. Using our formulation we can combine or chain together causal representations to solve specific problems without losing track of which assumptions are required to build these solutions, pushing real-world impact in healthcare, economics and business, environmental sciences and education, through causal deep learning.
BibTeX
@misc{berrevoets2023causal,
  author = {Jeroen Berrevoets and Krzysztof Kacprzyk and Zhaozhi Qian and Mihaela van der Schaar},
  title = {{Causal Deep Learning}},
  year = {2023},
  eprint = {2303.02186},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2303.02186},
}
Figure