Causal Duration Analysis with Diff-in-Diff

Ben Deaner, Hyejin KuarXiv 2024 · DOI · arXiv

Abstract: In economic program evaluation, it is common to obtain panel data in which outcomes are indicators that an individual has reached an absorbing state. For example, they may indicate whether an individual has exited a period of unemployment, passed an exam, left a marriage, or had their parole revoked. The parallel trends assumption that underpins difference-in-differences generally fails in such settings. We suggest identifying conditions that are analogous to those of difference-in-differences but apply to hazard rates rather than mean outcomes. These alternative assumptions motivate estimators that retain the simplicity and transparency of standard diff-in-diff, and we suggest analogous specification tests. Our approach can be adapted to general linear restrictions between the hazard rates of different groups, motivating duration analogues of the triple differences and synthetic control methods. We apply our procedures to examine the impact of a policy that increased the generosity of unemployment benefits, using a cross-cohort comparison.

Introduction

In policy evaluation, researchers often study "absorbing states"—events that, once they occur, do not "un-happen." Examples include exiting unemployment, passing a certification exam, or getting a divorce. To estimate the causal impact of a policy on these events, researchers frequently use the Difference-in-Differences (Diff-in-Diff) method. This approach relies on the "parallel trends" assumption: the idea that, without the policy, the gap between the treatment and control groups' average outcomes would have remained constant over time.

However, for absorbing states, this assumption is mathematically problematic. As time passes, more individuals in both groups eventually reach the state (e.g., 100% of students eventually pass a basic exam). Because outcomes are bounded at 100%, the groups' means must eventually converge. This mechanical convergence looks like a trend even when none exists, leading standard Diff-in-Diff to produce biased and misleading results—often underestimating the true impact of a policy.

Methodology: Hazard Diff-in-Diff

This paper proposes a solution: Hazard Diff-in-Diff. Instead of assuming parallel trends in the probability of having reached a state, the authors assume parallel trends in the hazard rate—the rate at which individuals who haven't yet reached the state finally do so.

Key Technical Features

  • Time-Average Hazard: The authors define a "time-average hazard" function calculated directly from observed mean outcomes using a logarithmic transformation. This allows for Diff-in-Diff estimation on these transformed values.
  • Simplicity: Unlike traditional duration models (such as Cox Proportional Hazards), this method does not require complex maximum likelihood estimation (MLE) or specific functional form specifications. It uses a "plug-in" estimator that is as easy to compute as standard Diff-in-Diff.
  • Covariate Adjustment: The framework includes a propensity score weighting scheme to handle cases where treatment and control groups have different baseline characteristics.
  • Extensions: The approach is flexible enough to adapt other popular methods, such as Triple Differences and Synthetic Control, to duration data.

Empirical Results

Simulation Studies

The authors evaluated the method using Monte Carlo simulations where the true data-generating process follows a duration model. In these large-sample simulations, standard Diff-in-Diff was consistently biased. In contrast, the Hazard Diff-in-Diff estimator was unbiased, and its confidence intervals achieved the correct coverage.

Austrian Unemployment Policy

The method was applied to a dataset from Lalive et al. (2006) tracking Austrian workers aged 40–49. The study examined how an extension of unemployment benefits (from 30 to 39 weeks) affected unemployment duration. The Hazard Diff-in-Diff approach found a statistically significant negative impact of extended benefits on the rate of finding a job. The treatment effect was strongest immediately following the point where benefits would have originally expired. Furthermore, a pre-treatment parallel trends test failed to reject the hazard-based assumption, providing strong evidence for the validity of the approach.

Discussion and Limitations

The "Hazard Diff-in-Diff" approach restores the utility of visual inspection and placebo tests for duration data by "straightening out" the trends. However, some limitations remain:

  • Nuisance Parameters: The method requires identifying certain coefficients in the linear relationship. If there are very few pre-treatment periods, these may not be unique.
  • Inequality Constraints: If researchers impose constraints (like positive weights) that become binding, standard statistical tests may require more specialized inference techniques.
  • Propensity Score Bounds: For covariate-adjusted versions, propensity scores must be bounded away from zero to ensure weights remain stable.

Future research could extend this framework to settings with "competing risks," where individuals can exit to multiple different states, such as finding a job versus retiring.

Summary updated

Abstract

In economic program evaluation, it is common to obtain panel data in which outcomes are indicators that an individual has reached an absorbing state. For example, they may indicate whether an individual has exited a period of unemployment, passed an exam, left a marriage, or had their parole revoked. The parallel trends assumption that underpins difference-in-differences generally fails in such settings. We suggest identifying conditions that are analogous to those of difference-in-differences but apply to hazard rates rather than mean outcomes. These alternative assumptions motivate estimators that retain the simplicity and transparency of standard diff-in-diff, and we suggest analogous specification tests. Our approach can be adapted to general linear restrictions between the hazard rates of different groups, motivating duration analogues of the triple differences and synthetic control methods. We apply our procedures to examine the impact of a policy that increased the generosity of unemployment benefits, using a cross-cohort comparison.
BibTeX
@misc{deaner2024causal,
  author = {Ben Deaner and Hyejin Ku},
  title = {{Causal Duration Analysis with Diff-in-Diff}},
  year = {2024},
  eprint = {2405.05220},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2405.05220},
}
Figure