Multi-Cause Effect Estimation with Disentangled Confounder Representation

Jing Ma, Ruocheng Guo, Aidong Zhang, Jundong Li

Abstract

Estimating causal effects from multiple simultaneous treatments is a critical task in fields ranging from medicine to economics. A primary challenge is the presence of unobserved confounders that influence both treatment assignments and outcomes. We propose DIRECT (Disentangled multIple tReatment EffeCT estimation), a novel framework that leverages a Variational Autoencoder (VAE) to learn hierarchical disentangled representations of these latent variables. By separating confounders into macro-level clusters and micro-level independent factors, DIRECT achieves superior estimation accuracy and provides human-interpretable insights. Furthermore, our approach models joint treatment representations, enabling generalization to unseen treatments. Experimental results on synthetic and real-world datasets demonstrate that DIRECT significantly outperforms existing state-of-the-art causal inference models.

Introduction

In causal inference, unobserved confounders—hidden variables affecting both the cause (treatment) and the effect (outcome)—pose a significant obstacle to accurate effect estimation. For instance, in analyzing the impact of a cast on a movie's box office revenue, "genre" acts as a hidden confounder influencing both actor selection and commercial success.

Causal graph illustrating the relationship between unobserved confounders $Z_i$, multiple treatments $A_{i,j}$, and the outcome $Y_i$.

Existing methods often treat these confounders as "black box" representations, which lack interpretability and fail to capture the hierarchical nature of real-world variables. Moreover, treatments in multi-cause scenarios are rarely independent, and models must be able to generalize to new treatments not present in the training data.

Methodology: The DIRECT Framework

The DIRECT framework addresses these challenges by modeling the causal process through a structured VAE architecture. It consists of an inference network for latent variable discovery and a generation network for outcome prediction.

The DIRECT framework: an inference network learns latent confounder representations, and a generation network reconstructs treatment assignments and predicts outcomes.

Hierarchical Disentanglement

DIRECT introduces a two-level disentanglement strategy:

  1. Macro-level: Treatments are grouped into clusters (e.g., "action" vs. "comedy" actors). Confounder representations are learned specifically for each cluster, allowing the model to specialize its understanding of latent factors.
  2. Micro-level: Within each representation, individual dimensions are forced to capture isolated factors of variation. This is achieved using a $\beta$-VAE approach, which applies a penalty to minimize correlation between dimensions, ensuring that each dimension represents a distinct semantic concept.

Joint Treatment Representation and Generalization

Unlike models that treat each treatment in isolation, DIRECT learns a shared function to represent treatments based on their assignment patterns. This captures dependencies—such as actors who frequently appear together—and enables zero-shot generalization. The model can estimate the effects of new treatments by observing their initial assignment patterns without requiring full retraining.

Evaluation and Results

We evaluated DIRECT on synthetic data and semi-synthetic datasets derived from Amazon product reviews (Amazon-3C and Amazon-6C). In these datasets, treatments are keywords in reviews, and outcomes are simulated sales figures.

Performance and Accuracy

DIRECT consistently outperformed traditional regression baselines (OLS, Random Forest) and specialized causal models like CEVAE and the Deconfounder. On the Amazon-6C dataset, DIRECT achieved a PEHE (Precision in Estimation of Heterogeneous Effect) of 5.04, a substantial improvement over the next best baseline, Deconf-l, which scored 8.16.

Interpretability and Insights

The disentangled representation allows for meaningful interpretation of the latent space. By manipulating specific dimensions of the learned confounder, we observed targeted effects on related treatments. For example, adjusting one dimension specifically influenced treatments associated with "musical instrument tuning," confirming that the model had isolated a "musical instrument" factor. Additionally, the macro-level clustering accurately reconstructed the ground-truth categories in synthetic tests.

Conclusion

DIRECT advances multi-cause effect estimation by introducing hierarchical disentanglement to latent confounder modeling. By combining macro-level clustering with micro-level factor isolation, the framework provides both high accuracy and interpretability. While the current model assumes a clusterable treatment structure, future work will explore more complex hierarchies and automated parameter tuning to further enhance its applicability to diverse real-world causal discovery tasks.

Summary updated

Abstract

One fundamental problem in causality learning is to estimate the causal effects of one or multiple treatments (e.g., medicines in the prescription) on an important outcome (e.g., cure of a disease). One major challenge of causal effect estimation is the existence of unobserved confounders -- the unobserved variables that affect both the treatments and the outcome. Recent studies have shown that by modeling how instances are assigned with different treatments together, the patterns of unobserved confounders can be captured through their learned latent representations. However, the interpretability of the representations in these works is limited. In this paper, we focus on the multi-cause effect estimation problem from a new perspective by learning disentangled representations of confounders. The disentangled representations not only facilitate the treatment effect estimation but also strengthen the understanding of causality learning process. Experimental results on both synthetic and real-world datasets show the superiority of our proposed framework from different aspects.
BibTeX
@inproceedings{ma2021multi,
  author = {Jing Ma and Ruocheng Guo and Aidong Zhang and Jundong Li},
  title = {{Multi-Cause Effect Estimation with Disentangled Confounder Representation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.24963/ijcai.2021/384},
}
Figure