TL;DR
This paper introduces a "map of causal deep learning" (CDL) to organize a rapidly growing and often confusing field. By refining Pearl’s ladder of causation and adding a second dimension for parametric assumptions, the authors provide a framework to categorize models by their structural knowledge and functional flexibility, helping researchers identify "white space" in the literature and practitioners select the right tools for their data.
The Problem: An Avalanche of Research
Deep learning (DL) excels at finding patterns in high-dimensional data but often lacks the ability to reason about cause and effect. Conversely, traditional causal inference provides rigorous frameworks like Structural Causal Models (SCMs) but often relies on simple, rigid functional forms.
The emerging field of Causal Deep Learning (CDL) attempts to marry these two worlds. However, the field is currently an "avalanche" of papers that are difficult to compare. A Transformer-based model and a linear regression model might both address "association," but they sit at opposite ends of the complexity spectrum. Existing frameworks, like Judea Pearl’s "ladder of causation," focus solely on causal rungs (association, intervention, counterfactuals) and ignore the "deep" part of deep learning—the parametric flexibility and representation space.
The Two-Axis Framework
The authors propose a two-axis "map" to categorize any CDL method based on its assumptions and capabilities:
- The Structural Scale (Vertical Axis): Measures how much we know about the relationships between variables.
- Level 1 (Unknown): No prior knowledge; variables are assumed to be potentially dependent.
- Level 2 (Plausible): Knowledge of statistical independencies (e.g., Probabilistic Graphical Models), but exact causal direction might be unknown. The authors call this "Rung 1.5."
- Level 3 (Causal): The exact causal structure (DAG) is known.
- The Parametric Scale (Horizontal Axis): Measures the assumptions made about the functions connecting variables.
- Non-parametric: No assumptions on functional form (the strength of Deep Learning).
- Noise Models: Assumptions on how noise enters the system (e.g., Additive Noise Models).
- Fully Parametric: Specific forms like linear or $n$-times differentiable functions.
- Fully Known: The functions are already known (no learning required).
The Map of CDL
The "Map" applies these scales to both the Input (what the model assumes) and the Representation (what the model learns or outputs). This distinction is crucial: a model might take non-parametric data as input but output a highly structured causal representation.
Key Insights and Research Gaps
- The "Rung 1.5" Concept: The authors argue that Pearl’s first rung (Association) is too broad for machine learning. They distinguish between models that assume nothing and models that encode statistical independencies, placing the latter at a hypothetical "Rung 1.5."
- The Pillars of CDL: CDL is defined by the interaction between leveraging structure (causality's strength) and non-parametric estimation (deep learning's strength). These are often in conflict; more structure usually requires stricter parametric assumptions to remain identifiable.
- Identifiability vs. Flexibility: The map visually demonstrates that the "holy grail"—mapping completely unknown, non-parametric input to a fully causal, non-parametric representation—is widely considered impossible without some form of inductive bias.
- Identifying Research Gaps: By plotting existing papers, the authors show that certain areas are overcrowded (e.g., learning structure in additive noise settings), while others are virtually empty (e.g., fully known parametric models, which don't require learning).
Conclusion
The primary result is the Map of CDL itself. The authors demonstrate its utility by categorizing high-profile supervised learning papers and framing the work as a "living paper." They intend to host an online forum where the community can discuss and propose changes to the map as the field evolves, providing a much-needed navigation system for the future of causal AI.
Summary updated