Causal Effect Inference for Structured Treatments

Jean Kaddour, Yuchen Zhu, Qi Liu, Matt J. Kusner, Ricardo Silva

This NeurIPS 2021 paper by Kaddour et al. tackles a frontier in causal inference: moving beyond simple binary or scalar treatments to complex, structured ones like molecules, text, or images. By introducing a deep learning framework called Structured Intervention Networks (SIN), the authors provide a robust method for estimating causal effects when treatments possess intricate internal structures.

The Challenge of Structured Treatments

Most causal inference research focuses on Conditional Average Treatment Effects (CATEs) for simple treatments—for example, evaluating whether a patient recovers better if given Drug A versus Drug B. However, in many real-world domains, treatments are inherently structured:

  • Medicine: A treatment is a complex molecular graph.
  • Nutrition: A treatment is a text-based food label.
  • Policy: A treatment is a geographic map of affected regions.
Problem Setup

Standard causal methods struggle in these scenarios. They cannot easily generalize to "unseen" treatments (like newly synthesized molecules) or handle the high-dimensional confounding between a subject's features and the complex treatment assigned to them. Without a way to "partial out" these correlations, models suffer from regularization bias, where the pressure to predict the outcome accurately causes the model to ignore the actual causal effect of the treatment.

Methodology: Generalized Robinson Decomposition & SIN

The paper's core contribution is the Generalized Robinson Decomposition (GRD) and its practical implementation, SIN.

  1. The Product Effect Assumption: The authors assume the outcome is a "product effect"—an inner product between covariate features $g(x)$ and treatment features $h(t)$. They prove this is a "universal" assumption; given enough dimensions, it can approximate any bounded continuous function.
  2. Generalized Robinson Decomposition (GRD): This mathematical reformulation isolates the causal estimand. It subtracts out "nuisance components": the mean outcome (what happens on average regardless of treatment) and "propensity features" (the expected treatment structure given the individual's features).
  3. Structured Intervention Networks (SIN): To implement GRD, the authors propose a two-stage training algorithm.
SIN Training Algorithm

As shown in the algorithm above, SIN operates in two stages:

  • Stage 1: Learns the mean outcome using standard supervised learning.
  • Stage 2: Simultaneously learns feature maps for covariates and treatments (using architectures like Graph Neural Networks) while alternating with a step that learns the propensity features to account for selection bias.

Datasets and Key Results

The authors evaluated their method on two primary simulations designed for structured treatments:

  • Small-World (SW): A synthetic dataset using 200 Watts-Strogatz small-world graphs as treatments and 1,000 uniformly sampled covariates.
  • TCGA (The Cancer Genome Atlas): A semi-synthetic benchmark using real-world gene expression data (9,659 features) as covariates and 10,000 molecular graphs from the QM9 dataset as treatments.

To measure success, they introduced new evaluation metrics, UPEHE and WPEHE (Unweighted/Weighted Precision in Estimation of Heterogeneous Effect), adapted to handle pairs of arbitrary structured treatments.

SIN significantly outperformed existing baselines, including GraphITE (a prior state-of-the-art for graph treatments) and standard GNN regressors. On the TCGA out-of-sample task (generalizing to new patients), SIN achieved a WPEHE@6 error of 8.15, compared to 27.48 for GraphITE and 27.57 for a standard GNN.

Key Insights

  • Isolating the Causal Signal: By using the Robinson decomposition, SIN "partials out" the parts of the outcome that are purely due to the patient's history, allowing the model to focus exclusively on how the structure of the treatment changes the result.
  • Generalization to the Unseen: Because SIN learns a representation of the treatment (e.g., an embedding of a molecular graph), it can predict the effect of a drug that was never seen in the training data, provided it shares structural similarities with known drugs.
  • Robustness to Bias: In scenarios with high "selection bias" (where doctors only prescribe certain drugs to certain types of patients), standard regression models often fail to beat a "zero effect" baseline. SIN remains robust because it explicitly models the propensity features.
  • Theoretical Strength: The authors established a quasi-oracle convergence guarantee, meaning that as long as the nuisance models (mean outcome and propensity) are "good enough," the SIN estimator performs as well as if it knew the ground-truth confounding factors.

Limitations and Future Work

Like most CATE estimators, SIN relies on the "unconfoundedness" assumption—that there are no unobserved variables affecting both treatment and outcome. Additionally, the authors note that neural network initialization can impact final estimates, suggesting that combining GRD with kernel methods might provide more stable consistency in the future. Finally, while SIN handles many treatments well, future work could explore even larger-scale "treatment spaces" where the number of possible structures is effectively infinite.

Summary updated

Abstract

We address the estimation of conditional average treatment effects (CATEs) for structured treatments (e.g., graphs, images, texts). Given a weak condition on the effect, we propose the generalized Robinson decomposition, which (i) isolates the causal estimand (reducing regularization bias), (ii) allows one to plug in arbitrary models for learning, and (iii) possesses a quasi-oracle convergence guarantee under mild assumptions. In experiments with small-world and molecular graphs we demonstrate that our approach outperforms prior work in CATE estimation.
BibTeX
@inproceedings{kaddour2021causal,
  author = {Jean Kaddour and Yuchen Zhu and Qi Liu and Matt J. Kusner and Ricardo Silva},
  title = {{Causal Effect Inference for Structured Treatments}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=_8tNOxYH8g},
}
Figure