TL;DR
This paper introduces "doubly robust proximal synthetic controls," a new method for estimating treatment effects in panel data. It addresses limitations of traditional synthetic control methods by relaxing assumptions about correctly modeling the counterfactual outcome, offering a more robust approach that is consistent even if one of the underlying models is misspecified.
What Problem Does This Paper Address?
Estimating the causal effect of an intervention (like a policy or treatment) on a single unit (e.g., a country, a state, a company) using panel data is a common challenge in fields like economics and public health. Traditional "synthetic control" (SC) methods construct a "synthetic" version of the treated unit by taking a weighted average of control units' outcomes. This synthetic unit is then used to estimate what would have happened to the treated unit in the absence of the intervention. However, these methods often rely on strong assumptions: they assume that the counterfactual outcome generating mechanism can be correctly modeled and that the pre-treatment trajectories of the treated and synthetic units can be matched almost perfectly. When these assumptions are violated, the estimates can be biased. This paper aims to relax these strict modeling requirements by leveraging ideas from proximal causal inference.
Methods
The authors propose two novel nonparametric identification formulas for the average treatment effect (ATT) for the treated unit.
- Weighting-based method: This approach uses a "treatment confounding bridge function" to capture the relationship between pre-treatment control unit outcomes and the unobserved confounders. This function is then used to weight pre-treatment observations to impute post-treatment counterfactuals.
- Doubly Robust method: This method combines models for the counterfactual outcome and the weighting function. It's "doubly robust" because it remains consistent and asymptotically normal if at least one of these two models is correctly specified, even if the other is misspecified. This is a significant advantage as it doesn't require users to know which model is correct.
Both methods are framed within the proximal causal inference framework and utilize the concept of "covariate shift" to obtain identification results. They develop estimators based on these formulas using the generalized method of moments (GMM).
Datasets
The paper uses the following datasets for evaluation:
- Brazil Pneumococcal Conjugate Vaccine Data: Monthly hospitalizations due to all-cause pneumonia among children less than 12 months old in Brazil from 2003 to 2013.
- Florida Homicide Data: Monthly data from 16 states (including Florida) on public safety measures from January 1999 to December 2014, used to study the effect of Florida's "stand your ground" law.
- Kansas GDP Data: Quarterly log GDP per capita for all US states from 1990 to 2016, used to study the effect of an aggressive tax cut in Kansas.
Key Insights and Ideas
- Relaxing Strict Modeling Assumptions: The core innovation is moving away from requiring perfect pre-treatment matching or correctly specifying the counterfactual outcome model. This makes the method more applicable in real-world scenarios where these ideal conditions might not hold.
- Doubly Robustness as a Safety Net: The doubly robust estimator is particularly valuable. It provides a form of insurance against model misspecification, meaning that as long as one of the two key components (outcome model or weighting model) is correct, the results are still reliable. This is a significant improvement over methods that fail if any component is misspecified.
- Proximal Causal Inference for SC: Adapting concepts from proximal causal inference to the synthetic control setting provides a powerful new lens for understanding and developing SC methods. This allows for more flexible modeling of unobserved confounders.
- Covariate Shift for Identification: The introduction of covariate shift is a clever way to handle the identification problem in panel data settings where standard propensity score concepts are not directly applicable.
Key Results
- Simulation Performance: Simulations show that the proposed doubly robust method is consistent and asymptotically normal when at least one of the outcome or weighting models is correctly specified. In contrast, simpler methods (like OLS or those relying on only one model) are biased when their respective assumptions are violated.
- Empirical Application (Pneumonia Vaccine): The method is applied to evaluate the effectiveness of a pneumococcal conjugate vaccine in Brazil. The results suggest a significant decrease in hospitalizations after the vaccine introduction.
- Empirical Application (Florida Homicide): The study of Florida's "stand your ground" law indicates that proximal methods, especially the doubly robust ones, can outperform OLS and traditional SC methods in noisy environments where a perfect pre-treatment fit is difficult to achieve.
- Empirical Application (Kansas GDP): The analysis of the Kansas tax cut shows that doubly robust methods provide significant results and point estimates that are often twice those of traditional SC methods, suggesting a potentially more accurate estimation.
Limitations and Future Work
- Dimensionality: The paper notes that the doubly robust method might require estimating more parameters, which could lead to numerical instability in scenarios with very short time series.
- Numerical Instability: The authors mention experiencing numerical instabilities in their GMM implementation, particularly with certain parameterizations of the confounding bridge function. They suggest using analytic gradients and careful initialization to mitigate this.
- Post-Selection Issues: The paper highlights that post-selecting units for proxies (W and Z) can lead to larger magnitudes and standard errors, potentially due to selecting wrong models. They recommend using prior knowledge for proxy selection when possible.
- Detrending: While detrending can help make assumptions more plausible, it might lead to slightly anti-conservative inference. The authors note that accounting for time trends appropriately in their methods remains an open question.
- Future Work: The authors suggest that their identification results could enable the development of new, more flexible nonparametric or semiparametric SC methods.
Summary updated