In a seminal paper presented at AISTATS 2010, Hui Guo and A. Philip Dawid take a critical look at one of the most popular tools in causal inference: the propensity score. Using a decision-theoretic framework, the authors investigate whether reducing a large set of "sufficient covariates" down to a single propensity variable actually helps us estimate causal effects more accurately. Their findings challenge common wisdom, particularly in the context of linear models.
The Problem of Confounding
When estimating the Average Causal Effect (ACE) of a treatment from observational data, researchers must account for confounding: the reality that individuals receiving treatment may differ fundamentally from those who do not. To address this, we adjust for "sufficient covariates"—variables that, once accounted for, render the treatment assignment "ignorable."
As shown in Figure 1, the treatment $T$ is influenced by covariates $X$, which also affect the outcome $Y$. While standard regression can adjust for $X$, high-dimensional covariate sets often lead to instability. Propensity scores—the probability of receiving treatment given covariates—are frequently used to collapse this information into a single scalar, theoretically simplifying the estimation problem.
Methodology: A Decision-Theoretic Approach
Moving away from the traditional "potential outcomes" framework, Guo and Dawid utilize a decision-theoretic framework. They define causal effects by comparing different regimes: an observational regime where data occurs naturally, and interventional regimes where treatment is forced to a specific value.
Within this framework, they analyze two types of covariate reduction in Normal Linear Models:
- Response-sufficient reduction: Reducing covariates based on their ability to predict the outcome (prognostic scores or "linear predictors").
- Treatment-sufficient reduction: Reducing covariates based on their ability to predict the treatment (propensity scores).
The Equivalence Paradox
The paper's most striking theoretical contribution is the "Equivalence Paradox." The authors prove (Theorem 4) that if one uses linear regression to estimate a propensity score ($LD^*$) and then uses that score in a subsequent linear regression to find the treatment effect, the result is mathematically identical to a standard regression on the full set of original covariates. In this context, the perceived "reduction" is a mathematical illusion that offers no gain in precision.
Simulation Results and Precision Rankings
To validate these theoretical insights, the authors conducted simulations across various scenarios, including homoscedastic and heteroscedastic cases. Figure 4 illustrates the distribution of estimated treatment coefficients across 25 simulated datasets.
Several key insights emerge from these results:
- The Equivalence of M0 and M3: The histograms for M0 (regression on $T$ and $X$) and M3 (regression on $T$ and the estimated propensity score $LD^*$) are identical, confirming the theoretical equivalence.
- The "Overfitting" Benefit: Surprisingly, adjusting for the true (known) propensity score (M2) resulted in significantly less precise estimates (higher MSE and SD) than using the estimated score (M3). The estimated score captures "accidental" balance within a specific sample, which actually reduces variance.
- Prognostic Superiority: Adjusting for the prognostic score (M1, the linear predictor of the outcome) proved to be the most effective strategy for increasing precision, outperforming both full regression and propensity-based methods.
Conclusion
Guo and Dawid's analysis suggests that the "propensity score hype" may be overblown for researchers primarily concerned with the precision of causal estimates in linear settings. While propensity scores are useful for dimensionality reduction, they do not inherently improve efficiency over standard regression in linear models. Furthermore, the study highlights that focusing on prognostic scores—how covariates predict the outcome—is often a more robust path toward precise causal inference.
Summary updated