Abstract
The Synthetic Control (SC) method is a cornerstone of policy evaluation, yet it relies on the often-fragile assumption that a treated unit can be perfectly represented as a linear combination of untreated donor units. This paper addresses the risks of model misspecification when this assumption fails. By introducing theoretical bounds—the M-bound and James-bound—and leveraging external demographic data, the authors demonstrate how to quantify estimation error and maintain robust causal inference even when the underlying linear model is imperfect.
The Problem: Fragile Linear Assumptions
In typical policy evaluation, researchers use the Synthetic Control method to estimate counterfactuals. For example, to study the impact of a tobacco tax in California, one might create a "Synthetic California" by weighting other states to match California's pre-intervention smoking rates. However, this method assumes that California is a simple linear combination of other states. If California’s population possesses unique characteristics that a weighted average cannot capture, the resulting estimates can be biased and misleading. This paper investigates how to make SC robust when these mathematical assumptions do not perfectly align with reality.
Methodology: Matching Populations, Not Just Outcomes
To address misspecification, the authors shift the focus from matching observed outcome curves to matching underlying population distributions. They propose a model where unit-level outcomes are viewed as averages of individual behaviors driven by specific demographic traits.
Two primary theoretical bounds are introduced to quantify mismatch:
- The M-bound: Utilizes the Wasserstein distance to measure the difference between the target population and the synthetic one, proving that small misspecifications lead to only small errors.
- The James-bound: Combines the Wasserstein distance of observed traits (like age and sex) with pre-intervention outcome fit to account for unobserved factors.
Unlike standard SC, which optimizes weights based solely on past outcomes, these new estimators prioritize weights that ensure the synthetic population is demographically similar to the target unit.
Key Results and the "Escape" Principle
The authors re-evaluated the classic 1988 California tobacco tax (Proposition 99) using their new framework. While they confirmed that California is not a perfect linear mix of other states, the calculated bounds showed that the observed drop in smoking was significantly larger than any potential error introduced by misspecification.
This illustrates the "Escape" principle: if the post-treatment data falls outside the calculated misspecification interval, it provides strong evidence that the policy had a genuine effect that cannot be attributed to model error. In synthetic experiments where the linear assumption was intentionally broken, the M-bound and James-bound estimators significantly outperformed standard SC and other common alternatives like ElasticNet.
Conclusion
By incorporating external demographic data as proxies for latent causes, this research provides a more rigorous foundation for the Synthetic Control method. While the approach requires high-quality external data, it offers practitioners a powerful tool to validate their findings against model misspecification, ensuring that policy evaluations remain reliable even in complex, non-linear real-world settings.
Summary updated