Balancing Reconstruction and Identification in Causal VAEs

This direction is merged into another direction: unobserved confounding (unmeasured confounding). Its last version stays readable and citable here.

Version 1 of 15 papers18 conceptsfrozen 2026-10-04

Estimating the effect of a treatment (like a drug) on an outcome (like recovery) is straightforward in a randomized trial. However, in observational data, "hidden confounders"—unobserved variables that influence both the treatment and the outcome—can bias our results.

To solve this, researchers use high-dimensional "proxies" (observed covariates like medical history or imaging) to infer these hidden factors. A popular tool for this is the Variational Autoencoder (VAE). But as recent research reveals, there is a fundamental tension between a VAE’s ability to reconstruct data and its ability to estimate causal effects.

The Conflict Between Reconstruction and Causality

In a standard VAE, the model learns a latent representation $Z$ by maximizing the Evidence Lower Bound (ELBO). This objective forces the model to do two things: represent the hidden confounder and reconstruct the observed input data as accurately as possible.

The problem, as identified by Severi Rissanen and Pekka Marttinen in A Critical Look at the Consistency of Causal Estimation with Deep Latent Variable Models, is that these two goals often compete. If the proxy data is complex (e.g., high-resolution images), the reconstruction loss will dominate the total objective function. The model becomes "distracted" by the need to perfectly recreate every pixel, causing it to ignore the subtle causal signals necessary to estimate the treatment effect.

This isn't just a matter of training time; it is a theoretical misalignment. Hisashi Kashima and Shonosuke Harada demonstrate in InfoCEVAE: treatment effect estimation with hidden confounding variables matching that reaching the mathematical global optimum of the ELBO does not guarantee the recovery of the correct Average Treatment Effect (ATE). A model can be a perfect autoencoder while remaining a poor causal estimator.

Prioritizing the Causal Signal

To fix this, researchers have moved toward "de-emphasizing" the autoencoding aspect of the model to let the causal relationship shine through.

One primary mechanism is the use of a tunable scaling parameter on the reconstruction loss. By down-weighting the importance of recreating the input covariates, the model is forced to focus its latent space on the variables that actually bridge the gap between treatment and outcome. Yang Li employs this in the ContiVAE framework, showing that preventing the model from becoming a "pure autoencoder" significantly improves the accuracy of individual dose-response curves.

Beyond Simple Latent Spaces

While scaling the loss helps, the structure of the latent space itself must also be more sophisticated to handle complex causal scenarios:

  1. Information-Theoretic Alignment: InfoCEVAE introduces information theory and matching techniques to ensure the latent variables are actually capturing the confounding influence rather than just random noise from the proxies.
  2. Specialized Priors: In Estimating Individual Dose-Response Curves under Unobserved Confounders from Observational Data, Yang Li introduces a "Tilted Gaussian" prior. This allows the model to better capture how different individuals respond to continuous treatments (like varying drug dosages) by providing a more flexible mathematical foundation for the latent confounders.

The Path to Consistency

The overarching lesson from this research direction is that causal inference requires more than just "plug-and-play" generative modeling. As Rissanen and Marttinen warn, these models can be highly sensitive to how the latent variables are specified.

The shift in the field is away from maximizing data fidelity and toward identifiability—ensuring that the specific parameters we care about (the causal effects) are the ones the model is incentivized to learn, even if that means the model becomes a "worse" autoencoder of the original input data.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_balancing_reconstruction_and_identificat_a6e588c4,
  title = {Balancing Reconstruction and Identification in Causal VAEs},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/balancing-reconstruction-and-identification-in-causal-vaes/rdr_a6e588c4bc5143ecb90b6a71f26ce475},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_balancing_reconstruction_and_identificat_a6e588c4_v1,
  title = {Balancing Reconstruction and Identification in Causal VAEs},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/balancing-reconstruction-and-identification-in-causal-vaes/rdr_a6e588c4bc5143ecb90b6a71f26ce475/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}