Doubly Robust Proximal Synthetic Controls

Hongxiang Qiu, Xu Shi, Wang Miao, Edgar Dobriban, Eric Tchetgen TchetgenBiometrics 2022

TL;DR

This paper introduces "doubly robust proximal synthetic controls," a new method for estimating treatment effects in panel data. It addresses limitations of traditional synthetic control methods by relaxing assumptions about correctly modeling the counterfactual outcome, offering a more robust approach that is consistent even if one of the underlying models is misspecified.

What Problem Does This Paper Address?

Estimating the causal effect of an intervention (like a policy or treatment) on a single unit (e.g., a country, a state, a company) using panel data is a common challenge in fields like economics and public health. Traditional "synthetic control" (SC) methods construct a "synthetic" version of the treated unit by taking a weighted average of control units' outcomes. This synthetic unit is then used to estimate what would have happened to the treated unit in the absence of the intervention. However, these methods often rely on strong assumptions: they assume that the counterfactual outcome generating mechanism can be correctly modeled and that the pre-treatment trajectories of the treated and synthetic units can be matched almost perfectly. When these assumptions are violated, the estimates can be biased. This paper aims to relax these strict modeling requirements by leveraging ideas from proximal causal inference.

Methods

The authors propose two novel nonparametric identification formulas for the average treatment effect (ATT) for the treated unit.

  1. Weighting-based method: This approach uses a "treatment confounding bridge function" to capture the relationship between pre-treatment control unit outcomes and the unobserved confounders. This function is then used to weight pre-treatment observations to impute post-treatment counterfactuals.
  2. Doubly Robust method: This method combines models for the counterfactual outcome and the weighting function. It's "doubly robust" because it remains consistent and asymptotically normal if at least one of these two models is correctly specified, even if the other is misspecified. This is a significant advantage as it doesn't require users to know which model is correct.

Both methods are framed within the proximal causal inference framework and utilize the concept of "covariate shift" to obtain identification results. They develop estimators based on these formulas using the generalized method of moments (GMM).

Datasets

The paper uses the following datasets for evaluation:

  • Brazil Pneumococcal Conjugate Vaccine Data: Monthly hospitalizations due to all-cause pneumonia among children less than 12 months old in Brazil from 2003 to 2013.
  • Florida Homicide Data: Monthly data from 16 states (including Florida) on public safety measures from January 1999 to December 2014, used to study the effect of Florida's "stand your ground" law.
  • Kansas GDP Data: Quarterly log GDP per capita for all US states from 1990 to 2016, used to study the effect of an aggressive tax cut in Kansas.

Key Insights and Ideas

  • Relaxing Strict Modeling Assumptions: The core innovation is moving away from requiring perfect pre-treatment matching or correctly specifying the counterfactual outcome model. This makes the method more applicable in real-world scenarios where these ideal conditions might not hold.
  • Doubly Robustness as a Safety Net: The doubly robust estimator is particularly valuable. It provides a form of insurance against model misspecification, meaning that as long as one of the two key components (outcome model or weighting model) is correct, the results are still reliable. This is a significant improvement over methods that fail if any component is misspecified.
  • Proximal Causal Inference for SC: Adapting concepts from proximal causal inference to the synthetic control setting provides a powerful new lens for understanding and developing SC methods. This allows for more flexible modeling of unobserved confounders.
  • Covariate Shift for Identification: The introduction of covariate shift is a clever way to handle the identification problem in panel data settings where standard propensity score concepts are not directly applicable.

Key Results

  • Simulation Performance: Simulations show that the proposed doubly robust method is consistent and asymptotically normal when at least one of the outcome or weighting models is correctly specified. In contrast, simpler methods (like OLS or those relying on only one model) are biased when their respective assumptions are violated.
  • Empirical Application (Pneumonia Vaccine): The method is applied to evaluate the effectiveness of a pneumococcal conjugate vaccine in Brazil. The results suggest a significant decrease in hospitalizations after the vaccine introduction.
  • Empirical Application (Florida Homicide): The study of Florida's "stand your ground" law indicates that proximal methods, especially the doubly robust ones, can outperform OLS and traditional SC methods in noisy environments where a perfect pre-treatment fit is difficult to achieve.
  • Empirical Application (Kansas GDP): The analysis of the Kansas tax cut shows that doubly robust methods provide significant results and point estimates that are often twice those of traditional SC methods, suggesting a potentially more accurate estimation.

Limitations and Future Work

  • Dimensionality: The paper notes that the doubly robust method might require estimating more parameters, which could lead to numerical instability in scenarios with very short time series.
  • Numerical Instability: The authors mention experiencing numerical instabilities in their GMM implementation, particularly with certain parameterizations of the confounding bridge function. They suggest using analytic gradients and careful initialization to mitigate this.
  • Post-Selection Issues: The paper highlights that post-selecting units for proxies (W and Z) can lead to larger magnitudes and standard errors, potentially due to selecting wrong models. They recommend using prior knowledge for proxy selection when possible.
  • Detrending: While detrending can help make assumptions more plausible, it might lead to slightly anti-conservative inference. The authors note that accounting for time trends appropriately in their methods remains an open question.
  • Future Work: The authors suggest that their identification results could enable the development of new, more flexible nonparametric or semiparametric SC methods.

Summary updated

Abstract

To infer the treatment effect for a single treated unit using panel data, synthetic control methods construct a linear combination of control units' outcomes that mimics the treated unit's pre-treatment outcome trajectory. This linear combination is subsequently used to impute the counterfactual outcomes of the treated unit had it not been treated in the post-treatment period, and used to estimate the treatment effect. Existing synthetic control methods rely on correctly modeling certain aspects of the counterfactual outcome generating mechanism and may require near-perfect matching of the pre-treatment trajectory. Inspired by proximal causal inference, we obtain two novel nonparametric identifying formulas for the average treatment effect for the treated unit: one is based on weighting, and the other combines models for the counterfactual outcome and the weighting function. We introduce the concept of covariate shift to synthetic controls to obtain these identification results conditional on the treatment assignment. We also develop two treatment effect estimators based on these two formulas and the generalized method of moments. One new estimator is doubly robust: it is consistent and asymptotically normal if at least one of the outcome and weighting models is correctly specified. We demonstrate the performance of the methods via simulations and apply them to evaluate the effectiveness of a Pneumococcal conjugate vaccine on the risk of all-cause pneumonia in Brazil.
BibTeX
@inproceedings{qiu2022doubly,
  author = {Hongxiang Qiu and Xu Shi and Wang Miao and Edgar Dobriban and Eric Tchetgen Tchetgen},
  title = {{Doubly Robust Proximal Synthetic Controls}},
  booktitle = {Biometrics},
  year = {2022},
}
Figure