Treatment effect estimation with disentangled latent factors

Weijia Zhang, Lin Liu, Jiuyong Li

Introduction

Estimating the causal effect of a treatment on an outcome is a fundamental challenge in fields ranging from medicine to public policy. While randomized controlled trials (RCTs) are the gold standard, researchers often must rely on observational data. In these settings, "confounders"—variables that influence both treatment assignment and the outcome—can introduce significant bias.

A common approach is to include all available covariates to satisfy the "unconfoundedness" assumption. However, this "kitchen sink" strategy can be counterproductive. Observational data often contains noisy proxies rather than direct measurements of true confounders. Furthermore, including "instrumental variables" (which only affect treatment) can increase bias and variance, while "risk factors" (which only affect the outcome) are useful for precision but are not confounders.

This paper introduces TEDVAE (Treatment Effect with Disentangled Autoencoder), a framework that leverages variational autoencoders to automatically disentangle latent factors from noisy proxies into three distinct categories: instrumental, risk, and confounding factors.

Methodology: The TEDVAE Framework

TEDVAE is a deep generative model designed to isolate relevant factors for treatment effect estimation. Unlike standard VAEs that learn a single latent representation, TEDVAE decomposes the latent space into three disjoint sets:

  1. Instrumental factors ($z_t$): Variables that influence the treatment $t$ but not the outcome $y$.
  2. Confounding factors ($z_c$): Variables that influence both $t$ and $y$.
  3. Risk factors ($z_y$): Variables that influence the outcome $y$ but not the treatment $t$.
The causal model diagram illustrating the disentanglement of latent factors $z_t$, $z_c$, and $z_y$ generating observed variables $x$, treatment $t$, and outcome $y$.

To achieve this disentanglement, the authors incorporate auxiliary classifiers into the VAE objective. One classifier ensures that $z_t$ and $z_c$ are predictive of the treatment, while another ensures $z_y$ and $z_c$ are predictive of the outcome. Crucially, the inference model depends only on the observed covariates $x$, making it more robust for predicting effects on new subjects without needing their treatment or outcome information during testing.

Experimental Evaluation

The authors evaluated TEDVAE against a suite of synthetic and real-world benchmarks, including the Infant Health Development Program (IHDP), the Twins dataset, and the 2016 Atlantic Causal Inference Challenge (ACIC).

Key Results

TEDVAE consistently outperformed state-of-the-art baselines, including Causal Forests, BART, and other deep learning models like CEVAE and GANITE.

  • ACIC 2016: TEDVAE achieved the lowest Precision in Estimation of Heterogeneous Effect (PEHE) of 1.75, outperforming 10 major baselines.
  • IHDP: The model reduced error by 30-40% compared to competing deep learning architectures.
  • Twins: For Average Treatment Effect (ATE) estimation, TEDVAE's error (0.006) was significantly lower than GANITE (0.016).
Performance comparison between TEDVAE and CEVAE across various data settings, highlighting TEDVAE's superior stability and lower error.

Discussion and Insights

The success of TEDVAE underscores the importance of active disentanglement in causal inference. By separating instrumental factors from confounders, the model avoids the pitfalls of over-adjustment. Empirical results show that TEDVAE is particularly robust to treatment imbalance—a common issue in observational studies where few subjects receive the treatment.

Furthermore, latent recovery analysis confirms that the model effectively utilizes its structured latent space. When the dimensions of specific factors are constrained, performance degrades, proving that the model relies on the identified disentangled components rather than just learning a generic high-dimensional representation.

Conclusion and Future Work

TEDVAE offers a robust solution for treatment effect estimation from high-dimensional, noisy observational data. While currently focused on binary treatments, the framework's theoretical foundations support extension to continuous treatments. Future work will explore these extensions and further refine the generative process for even more complex data distributions.

Summary updated

BibTeX
@inproceedings{zhang2021treatment,
  author = {Weijia Zhang and Lin Liu and Jiuyong Li},
  title = {{Treatment effect estimation with disentangled latent factors}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i12.17304},
}
Figure