TL;DR
This paper argues that a simple "naïve" regression is often as good as, or better than, the more complex "deconfounder" method for causal inference with multiple treatments. The authors demonstrate that the deconfounder's theoretical advantages only hold under extremely strong, often unrealistic, assumptions, and in practice, it frequently produces implausible results.
What Problem Does This Paper Address?
In observational studies, we often want to understand the causal effect of multiple "treatments" (e.g., different medications, policy interventions, or even actor appearances in movies) on an outcome. A major challenge is "confounding," where unobserved factors (Z) influence both the treatments (A) and the outcome (Y), leading to biased estimates. The "deconfounder" method, proposed by Wang and Blei (2019), uses factor models to estimate these unobserved confounders (Z) from the observed treatments (A) and then adjusts for them. This approach is popular in fields like genetics and political science. However, this paper questions the practical utility and theoretical underpinnings of the deconfounder, particularly its strong assumptions and performance compared to simpler methods.
Methods
The authors conduct a rigorous theoretical analysis of the deconfounder and its variants, focusing on asymptotic properties (how they behave with large sample sizes and many treatments). They prove that under certain conditions, a simple "naïve" regression (which ignores the confounder and directly regresses Y on A) is asymptotically unbiased. They also analyze various deconfounder estimators, including penalized and subset versions, showing that their advantages often disappear or require even stronger assumptions than the naïve approach. Crucially, they replicate existing deconfounder analyses and simulations, finding that the deconfounder rarely outperforms naïve regression and often produces implausible results.
Datasets
The paper uses the TMDB 5,000 dataset for the movie earnings case study. They also replicate simulations from previous work by Wang and Blei (2019) and Zhang et al. (2019), including medical health records, smoking data, and genetic data (GWAS).
Key Insights and Ideas
- Naïve Regression is Surprisingly Robust: The paper's core finding is that a simple linear regression of the outcome on the treatments (naïve regression) is often asymptotically unbiased under conditions where the deconfounder is claimed to work. This is because, with many treatments, the structure imposed by the confounder can be implicitly captured by the naïve regression.
- Deconfounder's Assumptions are Extreme: The deconfounder's "pinpointing" assumption, which is crucial for its theoretical guarantees, requires that each unobserved confounder affects an infinite number of treatments. This is a very strong and often unrealistic assumption in practice.
- Implausible Real-World Results: The deconfounder produced highly implausible estimates in the movie earnings case study, suggesting Stan Lee's cameos contributed billions to Marvel's revenue. This highlights the potential for the deconfounder to generate misleading conclusions when its assumptions are violated.
- Factor Model Machinery is Often Unnecessary: When the deconfounder does work (i.e., is estimable and asymptotically unbiased), it's often because it's essentially incorporating the same information as a naïve regression. The added complexity of the factor model machinery is frequently redundant.
- Posterior Predictive Checks are Unreliable: The authors show that posterior predictive checks, often used to validate deconfounder models, do not reliably indicate when the deconfounder will outperform simpler methods.
Key Results
- Asymptotic Unbiasedness: Under strong infinite confounding (where confounders affect infinitely many treatments), a naïve regression is asymptotically unbiased. Deconfounder variants are also asymptotically unbiased if they use the same information as the naïve regression.
- Inconsistency in Finite Settings: For any finite number of treatments, the deconfounder is shown to be inconsistent in nonlinear settings.
- Empirical Replication: Replications of deconfounder analyses and simulations consistently showed that the deconfounder did not outperform naïve regression, and often performed worse.
- Movie Earnings Case Study: The deconfounder estimated Stan Lee's cameos contributed $15.5 billion to Marvel movie revenue, a result deemed implausible. Adjusting for film budget yielded more reasonable estimates.
Limitations and Future Work
- Strong Assumptions: The paper emphasizes that the deconfounder relies on extremely strong and often untestable assumptions (e.g., infinite confounding, specific factor model structures, constant treatment effects). Relaxing these assumptions is a key area for future work.
- Real-World Applicability: The authors conclude that neither the deconfounder nor naïve regression are currently suitable for real-world applications without careful consideration of the underlying assumptions and research design. They suggest that more evidence is needed before these methods are widely adopted.
- Need for Better Model Checking: The paper highlights the limitations of current model checking techniques (like PPCs) in identifying when the deconfounder is performing well. Developing more reliable methods for assessing model fit and performance is an open question.
Summary updated