Abstract
Causal Deep Learning (CDL) is a unified framework designed to bridge the gap between the flexibility of deep learning and the rigor of causal inference. By organizing models along structural, parametric, and temporal dimensions, CDL provides a "topographic map" for researchers to navigate assumptions and identify research gaps. This framework enables the construction of robust pipelines for complex real-world applications in healthcare, economics, and beyond.
Introduction
Causal inference is often considered the "holy grail" of data science, as it seeks to understand the underlying mechanisms of data generation rather than mere correlations. However, a significant "practitioner's dilemma" exists: traditional causal methods often require extensive, untestable prior knowledge (e.g., a complete causal graph), while standard deep learning excels at high-dimensional data but lacks robustness to environmental changes.
Currently, there is no standardized language to describe the degree of "causality" in a deep learning model or to safely chain models together without violating underlying assumptions. CDL addresses this by providing a systematic way to "fingerprint" models based on their assumptions and capabilities.
The Causal Deep Learning Framework
The core of CDL is a three-dimensional framework that categorizes models based on the knowledge they assume or discover.
1. The Structural Dimension
The structural dimension measures the extent of knowledge regarding the links between variables. It spans three levels:
- Unknown (Level 1): Zero prior knowledge of the structure.
- Plausible (Level 2): Knowledge of statistical independencies or Probabilistic Graphical Models (PGMs).
- Causal (Level 3): Knowledge of the exact directed causal graph.
This dimension introduces a "Rung 1.5" to Pearl’s Ladder of Causality, capturing testable statistical constraints that do not fully define a causal structure but provide more information than simple association.
2. The Parametric Dimension
The parametric dimension describes the functional form of relationships between variables:
- Non-parametric (Level 1): Total flexibility in functional form.
- Noise Models (Level 2): Assumptions about noise distributions (e.g., additive noise).
- Fully Parametric (Level 3): Specific parametric forms are assumed.
- Fully Known (Level 4): The exact physical equations governing the system are known.
3. The Temporal Dimension
This dimension indicates whether a model accounts for the temporal ordering of cause and effect, which is a fundamental constraint in causal reasoning.
Knowledge Transitions and Model Chaining
A key concept in CDL is the Knowledge Transition. A model performs a transition if its output provides higher-level knowledge than its input. For example, a causal discovery algorithm that takes "Unknown" structure and outputs a "Plausible" graph performs a structural transition.
This framework allows for Safe Model Chaining. By mapping models on these axes, practitioners can ensure that the output of one model (e.g., a graph learner) satisfies the input requirements of another (e.g., a treatment effect estimator requiring a known graph).
Categorizing the Machine Learning Landscape
The CDL framework reveals distinct patterns in current research:
- Supervised Learning: Typically occupies the "Unknown Structure / Non-parametric" region, prioritizing predictive accuracy over causal understanding.
- Treatment Effects: Research has successfully relaxed parametric assumptions but often relies on a priori known causal structures.
- Reinforcement Learning (RL): RL and Bandits are more adept at structural transitions because they can resolve causal ambiguity through active interventions.
- Generative Modeling: While powerful, many causal generative models assume a fully known causal graph as an input, limiting their "wild" applicability.
Discussion and Future Directions
The CDL framework highlights several "white spaces" in current research. Notably, there is a lack of deep learning models that incorporate "Fully Known" parametric factors, such as physical laws. Furthermore, the transition from "Unknown/Non-parametric" data to a "Causal/Non-parametric" representation remains a significant challenge without interventions.
The authors urge the community to "fingerprint" their work using these dimensions. This transparency regarding assumptions is crucial for the judicious adoption of causal models in high-stakes fields like healthcare and economics, where validation remains a complex but essential task.
Summary updated