TL;DR
This paper introduces a new "design-robust" estimator for panel data that improves upon standard two-way fixed-effects (TWFE) regression. By incorporating information about how treatment is assigned, the new estimator is more robust to misspecification of either the treatment assignment model or the outcome model, offering a more reliable way to estimate causal effects in complex settings.
What Problem Does This Paper Address?
Estimating causal effects from panel data often relies on two-way fixed-effects (TWFE) regression. While transparent and simple, TWFE estimators can be problematic, especially when treatment assignment is not random or when treatment patterns are complex (like staggered adoption). Specifically, TWFE can lead to negative weights for certain treatment effects, making interpretation difficult and potentially leading to biased results. The authors aim to address this by developing an estimator that explicitly accounts for the treatment assignment mechanism, making it more robust and reliable, particularly in quasi-experimental settings where neither the assignment nor the outcome model is perfectly known.
Methods
The core of the proposed method is to augment the standard TWFE regression with unit-specific weights derived from a model of the treatment assignment mechanism. This approach is termed "design-robust" because it leverages the "design" of how treatment is assigned. The authors construct "oracle weights" based on a known assignment model and then show how to estimate these weights when the assignment model is unknown. A key feature is the "double robustness" property: the estimator remains consistent if either the assignment model or the outcome model is correctly specified. This is a significant improvement over standard TWFE, which often implicitly assumes a specific structure for both. They also introduce a "Reshaped Inverse Propensity Weighted (RIPW)" estimator and explore its properties, including consistency and asymptotic normality.
Datasets
The paper uses a synthetic dataset for simulation studies to evaluate the performance of their proposed estimator against standard TWFE and IPW estimators under various scenarios, including violations of parallel trends and heterogeneous treatment effects.
The paper also analyzes real-world data from an OpenTable dataset concerning the impact of state-level emergency declarations on restaurant reservations in the early COVID-19 pandemic. This dataset includes daily state-level year-over-year percentage changes in seated diners, indicators of state of emergency declarations, accumulated confirmed cases, vote share of Democrats, and number of hospital beds.
Key Insights and Ideas
- Leveraging the Assignment Mechanism is Crucial: The paper strongly advocates for explicitly modeling the treatment assignment process, even in observational studies. This information, often implicitly used or ignored, can significantly improve the robustness and reliability of causal effect estimates.
- Double Robustness as a Practical Advantage: The double robustness property is a major selling point. It means researchers don't need to perfectly specify both the treatment assignment and outcome models to get a consistent estimate. This is highly practical, as perfect specification is rare in real-world data.
- Addressing TWFE's Weaknesses: The proposed estimator directly tackles some of the well-documented issues with TWFE, such as negative weights, by reweighting observations based on the assignment mechanism.
- Generalizability Beyond Staggered Adoption: While the staggered adoption setting is a key motivating example, the methods are designed to be applicable to a broader range of general treatment assignment patterns.
Key Results
- The proposed design-robust estimator is shown to be more robust than the conventional TWFE estimator, remaining consistent if either the assignment mechanism or the TWFE regression model is correctly specified.
- In simulations, the RIPW estimator is demonstrated to be unbiased across various settings, including violations of parallel trends and heterogeneous treatment effects, where unweighted TWFE and IPW estimators can be biased.
- The analysis of the OpenTable dataset yields a significant negative treatment effect estimate for the state of emergency declaration on restaurant reservations, suggesting a substantial short-term impact.
Limitations and Future Work
- Static Models: The primary analysis focuses on static models, ruling out dynamic treatment effects. While an extension to handle dynamic effects is discussed, it's noted as beyond the scope of the main analysis.
- Parametric Assumptions: For inference when models are misspecified, the paper relies on parametric assumptions for fitting assignment and outcome models.
- Computational Complexity: The proposed methods, particularly the cross-fitting and de-randomization procedures, can be computationally intensive.
- Future Research: The authors suggest exploring extensions to dynamic treatment effects and developing more efficient estimators. They also note that the analysis of dependent assignments for cross-fitting estimators is an area for future work.
Summary updated