Operationalizing Complex Causes: A Pragmatic View of Mediation

Limor Gultchin, David S. Watson, Matt J. Kusner, Ricardo Silva

Abstract

Traditional causal inference often relies on "atomic" interventions that are difficult to implement in complex systems like genomics or natural language processing. This paper introduces a framework for "pragmatic mediation," which utilizes high-level features of complex objects to estimate causal effects from "crude" interventions. By exploiting the invariant relationship between these mediators and the final outcome, the proposed method enables accurate outcome prediction for new treatment regimes even when labeled data is extremely scarce.

Introduction: The Challenge of Crude Interventions

In classical causal inference, we often conceptualize "surgical" interventions—the ability to set a single variable to a specific value ($do(x)$) while keeping all other factors constant. However, in many real-world domains, interventions are inherently "crude." For instance, a gene therapy ($W$) affects a complex biological process ($X$), which then influences disease activity ($Y$). We cannot easily isolate the effect of a single gene without triggering a cascade of secondary effects.

This complexity introduces two primary challenges:

  1. Ambiguity: The direct "atomic" effect of $X$ on $Y$ may be ill-defined or impossible to measure in isolation.
  2. Data Scarcity: When a new treatment ($w^*$) is introduced, we typically have very few samples where the final outcome $Y$ has been measured, rendering standard supervised learning approaches ineffective.
Conceptual overview of pragmatic mediation, contrasting historical data with limited new data regimes.

Methodology: Pragmatic Mediation

The authors propose a two-stage approach that leverages the stability of the relationship between the complex object $X$ and the outcome $Y$ across different treatment regimes.

1. Causal Response Estimation

The process begins by defining a "dictionary" of high-level features $\phi(X, Z)$, which serve as candidate mediators.

  • Stage 1: A model is trained to predict the expected value of these features given the treatment $W$ and covariates $Z$.
  • Stage 2: The outcome $Y$ is regressed on these predicted feature expectations using historical data.

Because the relationship between the mediators and the outcome is assumed to be invariant, this model can predict $Y$ for a novel treatment $w^*$ by observing only how $w^*$ affects the mediators $X$. This is often significantly easier and cheaper than waiting to observe the final outcome $Y$.

2. Mediation Discovery (The M-Criteria)

To identify the specific features driving the causal effect, the authors introduce the "M-Criteria." A feature is considered a pragmatic mediator if it satisfies two conditions:

  1. It must covary with the treatment $W$ (indicating it is affected by the intervention).
  2. It must have a "direct effect" on $Y$ (remaining associated with the outcome even after conditioning on other features).
A taxonomy for identifying causal shifts and pragmatic mediators based on conditional independence relations.

Experimental Evaluation

The framework was evaluated across three distinct domains:

  • ImagePert: A simulated dataset of pixel patterns used to provide visual intuition on how perturbations propagate through a system.
  • Humicroedit: A text dataset where users edit headlines for humor. The intervention is the edit, the complex object is the resulting headline (GloVe embeddings), and the outcome is a funniness score.
  • DREAM5: A semi-simulated genomics dataset based on the E. coli transcriptome, using "kernel eigengenes" to represent biological pathways.

Key Results

  • Low-Data Performance: In all experiments, the proposed method significantly outperformed standard baselines (Lasso, Random Forests, Gradient Boosting) when labeled data in the new treatment regime was limited (10-40%).
  • Discovery Accuracy: In the genomics task, the method successfully recovered 92% of true mediators with an overall accuracy of 85%, proving its utility for hypothesis generation.
  • Robustness: Performance remained consistent across over 100 different configurations of the target variable $Y$.

Discussion and Limitations

While powerful, the "pragmatic mediation" framework relies on several key assumptions. It assumes that $X$ fully mediates the effect of $W$ on $Y$; any significant direct effect from $W$ to $Y$ that bypasses the measured features of $X$ could bias predictions. Furthermore, the success of the method is contingent on a "rich enough" feature dictionary $\phi$ that captures the underlying causal mechanisms. Future work aims to extend this framework to handle unobserved confounding between $W$ and $Y$.

Summary updated

Abstract

We examine the problem of causal response estimation for complex objects (e.g., text, images, genomics). In this setting, classical \emph{atomic} interventions are often not available (e.g., changes to characters, pixels, DNA base-pairs). Instead, we only have access to indirect or \emph{crude} interventions (e.g., enrolling in a writing program, modifying a scene, applying a gene therapy). In this work, we formalize this problem and provide an initial solution. Given a collection of candidate mediators, we propose (a) a two-step method for predicting the causal responses of crude interventions; and (b) a testing procedure to identify mediators of crude interventions. We demonstrate, on a range of simulated and real-world-inspired examples, that our approach allows us to efficiently estimate the effect of crude interventions with limited data from new treatment regimes.
BibTeX
@inproceedings{gultchin2021operationalizing,
  author = {Limor Gultchin and David S. Watson and Matt J. Kusner and Ricardo Silva},
  title = {{Operationalizing Complex Causes: A Pragmatic View of Mediation}},
  booktitle = {International Conference on Machine Learning},
  year = {2021},
  url = {https://openreview.net/forum?id=t9CBJYvUcT4},
}
Figure