Causal Effect Inference for Structured Treatments

Jean Kaddour, Yuchen Zhu, Qi Liu, Matt J. Kusner, Ricardo Silva

Abstract

We address the estimation of conditional average treatment effects (CATEs) for structured treatments (e.g., graphs, images, texts). Given a weak condition on the effect, we propose the generalized Robinson decomposition, which (i) isolates the causal estimand (reducing regularization bias), (ii) allows one to plug in arbitrary models for learning, and (iii) possesses a quasi-oracle convergence guarantee under mild assumptions. In experiments with small-world and molecular graphs we demonstrate that our approach outperforms prior work in CATE estimation.

Introduction

Estimating the Conditional Average Treatment Effect (CATE) is a cornerstone of causal inference, answering the counterfactual question: "How would the outcome change for this specific individual if we applied a different treatment?" While traditional methods excel with binary or scalar treatments, modern applications often involve structured treatments—complex objects like molecular graphs in drug discovery, high-resolution images in medical imaging, or text in policy analysis.

Structured treatments present unique challenges. First, the space of possible treatments is effectively infinite (e.g., the space of all possible molecules), meaning we rarely observe the same treatment twice in a dataset. Second, confounding—where covariates influence both the treatment assignment and the outcome—makes it difficult to disentangle true causal effects from mere correlations. Finally, standard deep learning models often suffer from regularization bias, where the model prioritizes predicting the outcome from high-dimensional covariates and "ignores" the treatment signal, leading to biased effect estimates.

Overview of the causal graph, observed data, and the goal of estimating CATE for structured treatments.

Methodology: Generalized Robinson Decomposition

To overcome these hurdles, this research introduces the Generalized Robinson Decomposition (GRD). This framework bridges the gap between classical semi-parametric statistics and deep representation learning through two main contributions.

The Product Effect Assumption

The core of the GRD approach is the assumption of a product effect. The outcome equation is rewritten to isolate the causal effect, modeling it as an inner product between covariate features and treatment features. While this might seem restrictive, the authors prove that this formulation is a universal approximator: it can represent any continuous function provided the embedding dimensions are sufficiently large. This justifies the use of neural network embeddings to capture complex causal interactions.

Structured Intervention Networks (SIN)

The GRD is implemented through Structured Intervention Networks (SIN), a two-stage training procedure designed to isolate the causal signal from confounding noise.

  1. Stage 1: Nuisance Modeling. The model first learns to predict the mean outcome based solely on covariates. This "partials out" the portion of the outcome that can be explained without considering the treatment, effectively performing targeted regularization.
  2. Stage 2: Residual Learning. The model uses the residuals from Stage 1 to learn joint embeddings for the structured treatments and covariates. By focusing on the residuals, SIN prevents the treatment effect from being "washed out" by the covariate signal.

Additionally, SIN learns "propensity features"—covariate representations that explain treatment assignment—which is a more tractable alternative to estimating traditional propensity scores in high-dimensional structured spaces.

Training algorithm and pseudocode for Structured Intervention Networks (SIN).

Experimental Evaluation

The authors evaluated SIN using two primary simulation environments where ground-truth causal effects are known:

  • Small-World (SW): A synthetic setup using 1,000 units and 200 Watts-Strogatz small-world graphs as treatments.
  • TCGA (The Cancer Genome Atlas): A semi-synthetic task using real gene expression data (covariates) from cancer patients paired with 10,000 molecular graphs from the QM9 dataset as treatments.

To measure performance, the study introduced UPEHE and WPEHE (Unweighted/Weighted Precision in Estimation of Heterogeneous Effect), metrics specifically adapted for the structured treatment setting.

Key Results

Experiments demonstrate that SIN significantly outperforms existing baselines, including GraphITE and standard GNN regressors:

  • Superior Accuracy: In the TCGA molecular task, SIN achieved a reduction in error (WPEHE) of over 60% compared to the next best method.
  • Generalization: SIN showed a strong ability to predict effects for unseen treatments (out-sample), a critical requirement for tasks like virtual screening in drug discovery.
  • Robustness to Bias: As selection bias (confounding) increased, SIN maintained stable performance, whereas standard models degraded rapidly.

Conclusion and Future Work

SIN provides a robust, "plug-and-play" framework for causal inference with structured treatments. By isolating the causal estimand through the Generalized Robinson Decomposition, it successfully mitigates regularization bias and handles confounding in high-dimensional spaces.

Future research will focus on relaxing the "unconfoundedness" assumption by incorporating instrumental variables and addressing the sensitivity of deep causal models to hyperparameter initialization. Additionally, extending the theoretical "quasi-oracle" guarantees to the full end-to-end representation learning setting remains an important open challenge.

Summary updated

Abstract

We address the estimation of conditional average treatment effects (CATEs) for structured treatments (e.g., graphs, images, texts). Given a weak condition on the effect, we propose the generalized Robinson decomposition, which (i) isolates the causal estimand (reducing regularization bias), (ii) allows one to plug in arbitrary models for learning, and (iii) possesses a quasi-oracle convergence guarantee under mild assumptions. In experiments with small-world and molecular graphs we demonstrate that our approach outperforms prior work in CATE estimation.
BibTeX
@inproceedings{kaddour2021causal,
  author = {Jean Kaddour and Yuchen Zhu and Qi Liu and Matt J. Kusner and Ricardo Silva},
  title = {{Causal Effect Inference for Structured Treatments}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=_8tNOxYH8g},
}
Figure