Causal learning without DAGs

David Duvenaud, Daniel Eaton, Kevin P. Murphy, Mark SchmidtNeurIPS 2010 · OpenReview

Most causal discovery methods focus on finding a "true" Directed Acyclic Graph (DAG) to represent how variables interact. However, nature is often too messy for such simple structures. This paper argues that we should evaluate causal models by their ability to predict the effects of interventions on unseen data, demonstrating that non-DAG models—like undirected graphs and mixture models—often perform as well as or better than traditional DAGs.

The Problem with the DAG Assumption

In the world of causal inference, the Directed Acyclic Graph (DAG) is king. Researchers typically try to "discover" the graph structure that represents how variables cause one another. However, the authors point out a major flaw: in the real world, we rarely know the "ground truth" graph. Furthermore, biological and physical systems are full of feedback loops, time-varying dynamics, and hidden variables—features that a DAG, by definition, cannot represent.

The authors propose a shift in perspective. Instead of obsessing over whether a graph is "correct," they suggest a "black box" or functional view of causality. Under this view, a model is "causal" if it can accurately predict the state of a system after an intervention (an action, like a drug treatment or a gene knockout). This allows us to evaluate models objectively based on their predictive performance on test data, rather than their resemblance to a theoretical diagram.

Methodology: Models and Regimes

The researchers compared several classes of models to see which could best predict the effects of interventions, denoted as $p(x|a)$, where $x$ is the system state and $a$ is the action taken:

  1. DAG-based Models: Including the standard "perfect intervention" model (where an action completely overrides a variable) and "uncertain intervention" models (where an action might have "fat hand" side effects on multiple variables).
  2. Undirected Graphical Models (UGMs): Also known as Markov Random Fields. These model dependencies without assuming a direction of flow or an acyclic structure.
  3. Mixture Models: Specifically, mixtures of multinomials and logistic regressions. These serve as "graph-free" density estimators.

They tested these models under three training regimes:

  • Ignore: Pooling all data and ignoring which actions were taken.
  • Independent: Fitting a separate model for every unique combination of actions.
  • Conditional: Building a single model that takes the actions as input, allowing the model to "borrow strength" across different experimental conditions.

Datasets Evaluated

The paper evaluates these approaches on three distinct types of data:

  • T-cell Signaling Data (Sachs et al., 2005): A classic "gold standard" biological dataset measuring 11 proteins across thousands of individual cells under various chemical interventions.
  • DREAM 3 Phosphoprotein Data: Data from a 2008 competition involving 17 proteins and 58 different combinations of stimuli and inhibitors.
  • Synthetic Data: Data generated from Structural Equation Models (SEMs) that specifically included feedback loops to simulate systems where the DAG assumption is intentionally violated.

Key Findings and Results

Interventions as Features

By treating interventions as conditional inputs rather than just graph-cutting operations, models can learn how actions relate to one another. This is crucial for predicting the effect of a "double knockout" when you've only seen single knockouts. If the underlying system has feedback (common in biology), forcing it into a DAG structure actually hurts predictive accuracy.

T-cell Data Performance

In 10-fold cross-validation on the T-cell data, the "Conditional UGM" and "Conditional DAG" performed significantly better than models that ignored interventions. Interestingly, non-DAG models (like Mixture Models) were competitive with the best DAG-based methods.

Average Negative Log-Likelihood of different models. Conditional models significantly outperform models that ignore interventions and the 'perfect' DAG.

Predicting Novel Actions

When tasked with predicting the effect of an action never seen during training, the "Perfect Intervention" DAG—the theoretical ideal for this task—was actually slightly worse than conditional DAGs and UGMs. Undirected models, which are often easier to train and don't require acyclicity, performed remarkably well at predicting interventional effects, challenging the necessity of directed graphs for causal prediction.

Predictive performance on novel actions. Conditional UGMs and DAGs outperform the theoretical 'Perfect Intervention' DAG as training examples increase.

Simplicity Wins

In the DREAM 3 challenge, the authors found that simple linear regression and Probabilistic Matrix Factorization (PMF) outperformed all the complex graph-based entries submitted to the competition. Furthermore, on synthetic data with feedback loops, DAG models were fundamentally limited, while conditional non-DAG models maintained high predictive accuracy.

Limitations and Future Work

The authors acknowledge a significant hurdle: if an action is treated as an "atomic symbol" (e.g., "Action 7"), it is impossible to predict its effect if it has never been seen before without making strong assumptions (like the perfect intervention assumption).

They suggest that the future of causal learning lies in feature-based action descriptions. Instead of treating a drug as "Action A," we should describe it by its chemical properties or known targets. This would turn causal discovery into a structured prediction problem, allowing models to generalize to entirely new interventions based on the features of the action itself, rather than just its position in a graph.

Summary updated

Abstract

Causal learning methods are often evaluated in terms of their ability to discover a true underlying directed acyclic graph (DAG) structure. However, in general the true structure is unknown and may...
BibTeX
@inproceedings{duvenaud2010causal,
  author = {David Duvenaud and Daniel Eaton and Kevin P. Murphy and Mark Schmidt},
  title = {{Causal learning without DAGs}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2010},
  url = {https://openreview.net/forum?id=QwlaCGwn0b7},
}
Figure