Quantifying causal influences

Dominik Janzing, Moritz Grosse‐Wentrup, Bernhard Schölkopf, David BalduzziThe Annals of Statistics 2013 · DOI

TL;DR

This paper introduces a novel framework for quantifying causal influence between variables in a directed acyclic graph (DAG). Instead of just identifying causal relationships, it proposes a principled way to measure their strength using a communication-inspired scenario and relative entropy. This new measure satisfies several intuitive postulates that existing methods often fail, offering a more robust way to understand the impact of interventions.

What Problem Does This Paper Address?

In causal inference, we often represent relationships between variables using Directed Acyclic Graphs (DAGs). While a DAG tells us if one variable influences another, it doesn't tell us how much. Quantifying this "causal strength" is crucial for making informed decisions, especially when dealing with complex systems where some influences might be weak and others strong. Existing measures like Average Causal Effect (ACE), transfer entropy, and information flow have limitations; they can fail to satisfy basic logical requirements (postulates) when applied to simple causal structures. This paper aims to fill this gap by proposing a new measure of causal strength that is both theoretically sound and intuitively appealing.

Methods

The authors propose a novel approach to quantify causal strength based on a communication scenario. Imagine each edge in a DAG as a communication channel. Causal strength is then defined as the impact of "corrupting" or "cutting" these channels via interventions. Specifically, they define a "post-cutting" distribution by removing an edge and feeding the open end with the marginal distribution of the source variable. The causal strength of an arrow (or a set of arrows) is then measured by the Kullback-Leibler (KL) divergence between the original distribution and this new "post-cutting" distribution. This approach is then extended to structural equation models, where interventions involve introducing independent copies of variables.

Datasets

Not discussed in the paper. The paper focuses on theoretical development and uses simulated data for experiments.

Key Insights and Ideas

  • Causal Strength as "Channel Corruption": The core idea is that the strength of a causal link can be measured by how much the system's behavior changes when that link is disrupted. This is framed as corrupting a communication channel.
  • Postulates for Soundness: The authors propose five intuitive postulates (Causal Markov Condition, Mutual Information, Locality, Quantitative Causal Markov Condition, Heredity) that a good measure of causal strength should satisfy. They demonstrate how many existing measures fail these tests.
  • Relative Entropy as the Metric: The Kullback-Leibler divergence is chosen as the mathematical tool to quantify the difference between the original and "post-cutting" distributions, providing a principled way to measure the impact of interventions.
  • Distinguishing Observation from Intervention: The paper highlights the critical difference between simply observing data and performing interventions (e.g., using Pearl's do-calculus), and how their measure correctly accounts for this distinction.
  • Generalizability: The proposed measure is shown to be applicable to both general DAGs and linear structural equation models, and can quantify the strength of individual arrows or sets of arrows.

Key Results

  • The proposed measure of causal strength satisfies the five introduced postulates, unlike many existing methods like ACE, transfer entropy, and information flow, which are shown to fail on simple DAGs.
  • The measure is shown to be additive for arrows with different targets and exhibits a heredity property, aligning with intuitive expectations.
  • The paper demonstrates through examples that their measure correctly quantifies causal influence in scenarios where transfer entropy fails, particularly in cases of strong confounding or deterministic relationships.
  • For linear structural equations under Gaussianity, an explicit expression for causal strength is derived, relating it to covariance matrices.

Limitations and Future Work

  • Completeness of Postulates: The authors acknowledge that their list of postulates might not be exhaustive and that other measures might be appropriate depending on the specific research goal.
  • Practical Estimation: While the theoretical framework is presented, the paper touches upon the challenges of estimating causal strength from finite data, especially for high-dimensional distributions or when dealing with deterministic relationships where relative entropy can diverge.
  • Total Influence: The supplementary material mentions exploring the problem of defining the "total influence" of one set of nodes on another, suggesting this is a direction for future work.
  • Generalization: The paper focuses on DAGs and linear structural equations, with a brief mention of extensions to additive noise models and post-nonlinear models, indicating potential for broader applicability.

Summary updated

Abstract

Many methods for causal inference generate directed acyclic graphs (DAGs) that formalize causal relations between $n$ variables. Given the joint distribution on all these variables, the DAG contains all information about how intervening on one variable changes the distribution of the other $n-1$ variables. However, quantifying the causal influence of one variable on another one remains a nontrivial question. Here we propose a set of natural, intuitive postulates that a measure of causal strength should satisfy. We then introduce a communication scenario, where edges in a DAG play the role of channels that can be locally corrupted by interventions. Causal strength is then the relative entropy distance between the old and the new distribution. Many other measures of causal strength have been proposed, including average causal effect, transfer entropy, directed information, and information flow. We explain how they fail to satisfy the postulates on simple DAGs of $\leq3$ nodes. Finally, we investigate the behavior of our measure on time-series, supporting our claims with experiments on simulated data.
BibTeX
@article{janzing2013quantifying,
  author = {Dominik Janzing and Moritz Grosse‐Wentrup and Bernhard Schölkopf and David Balduzzi},
  title = {{Quantifying causal influences}},
  journal = {The Annals of Statistics},
  year = {2013},
  doi = {10.1214/13-aos1145},
}
Figure