Abstract
Estimating causal effects from multiple simultaneous treatments is a critical task in fields ranging from medicine to economics. A primary challenge is the presence of unobserved confounders that influence both treatment assignments and outcomes. We propose DIRECT (Disentangled multIple tReatment EffeCT estimation), a novel framework that leverages a Variational Autoencoder (VAE) to learn hierarchical disentangled representations of these latent variables. By separating confounders into macro-level clusters and micro-level independent factors, DIRECT achieves superior estimation accuracy and provides human-interpretable insights. Furthermore, our approach models joint treatment representations, enabling generalization to unseen treatments. Experimental results on synthetic and real-world datasets demonstrate that DIRECT significantly outperforms existing state-of-the-art causal inference models.
Introduction
In causal inference, unobserved confounders—hidden variables affecting both the cause (treatment) and the effect (outcome)—pose a significant obstacle to accurate effect estimation. For instance, in analyzing the impact of a cast on a movie's box office revenue, "genre" acts as a hidden confounder influencing both actor selection and commercial success.
Existing methods often treat these confounders as "black box" representations, which lack interpretability and fail to capture the hierarchical nature of real-world variables. Moreover, treatments in multi-cause scenarios are rarely independent, and models must be able to generalize to new treatments not present in the training data.
Methodology: The DIRECT Framework
The DIRECT framework addresses these challenges by modeling the causal process through a structured VAE architecture. It consists of an inference network for latent variable discovery and a generation network for outcome prediction.
Hierarchical Disentanglement
DIRECT introduces a two-level disentanglement strategy:
- Macro-level: Treatments are grouped into clusters (e.g., "action" vs. "comedy" actors). Confounder representations are learned specifically for each cluster, allowing the model to specialize its understanding of latent factors.
- Micro-level: Within each representation, individual dimensions are forced to capture isolated factors of variation. This is achieved using a $\beta$-VAE approach, which applies a penalty to minimize correlation between dimensions, ensuring that each dimension represents a distinct semantic concept.
Joint Treatment Representation and Generalization
Unlike models that treat each treatment in isolation, DIRECT learns a shared function to represent treatments based on their assignment patterns. This captures dependencies—such as actors who frequently appear together—and enables zero-shot generalization. The model can estimate the effects of new treatments by observing their initial assignment patterns without requiring full retraining.
Evaluation and Results
We evaluated DIRECT on synthetic data and semi-synthetic datasets derived from Amazon product reviews (Amazon-3C and Amazon-6C). In these datasets, treatments are keywords in reviews, and outcomes are simulated sales figures.
Performance and Accuracy
DIRECT consistently outperformed traditional regression baselines (OLS, Random Forest) and specialized causal models like CEVAE and the Deconfounder. On the Amazon-6C dataset, DIRECT achieved a PEHE (Precision in Estimation of Heterogeneous Effect) of 5.04, a substantial improvement over the next best baseline, Deconf-l, which scored 8.16.
Interpretability and Insights
The disentangled representation allows for meaningful interpretation of the latent space. By manipulating specific dimensions of the learned confounder, we observed targeted effects on related treatments. For example, adjusting one dimension specifically influenced treatments associated with "musical instrument tuning," confirming that the model had isolated a "musical instrument" factor. Additionally, the macro-level clustering accurately reconstructed the ground-truth categories in synthetic tests.
Conclusion
DIRECT advances multi-cause effect estimation by introducing hierarchical disentanglement to latent confounder modeling. By combining macro-level clustering with micro-level factor isolation, the framework provides both high accuracy and interpretability. While the current model assumes a clusterable treatment structure, future work will explore more complex hierarchies and automated parameter tuning to further enhance its applicability to diverse real-world causal discovery tasks.
Summary updated