Combining Panel Data Sets with Attrition and Refreshment Samples

Keisuke Hirano, Guido W. Imbens, Geert Ridder, Donald Rebin1998 · DOI

Abstract

With panel data, important issues can be resolved that cannot be addressed with cross-sectional data. A major drawback is that panel data suffer from more severe missing data problems. Adding a sample consisting of new units randomly drawn from the original population as replacements for units who have dropped out of the panel—a so-called refreshment sample—can be helpful in mitigating the effects of attrition, both by allowing for estimation of richer models and by making estimation of conventional models more precise. In this paper, we develop a family of models that incorporate refreshment samples, and we demonstrate in an application to a Dutch data set on travel behavior that such models can lead to substantially different results than models that assume that the missing data process is ignorable or conventional econometric models for panel data with attrition.

Introduction

In economics and social sciences, panel data is highly valued for its ability to follow the same individuals over time, allowing researchers to control for unobserved individual characteristics. However, longitudinal studies frequently suffer from "attrition"—the tendency for participants to drop out over successive waves. If this dropout process is non-random (e.g., high-income individuals being too busy to respond), the remaining sample becomes biased, leading to incorrect inferences.

Traditionally, researchers have relied on two primary, often untestable, assumptions to address missing data:

  1. Missing at Random (MAR): Assumes that dropouts depend only on data collected in previous waves.
  2. Hausman-Wise (HW): Assumes that dropouts depend on current, unobserved values.

This paper introduces a more flexible approach using "refreshment samples"—separate, random samples taken at the same time as later panel waves—to detect and correct for non-random attrition.

Methodology: The Additive Nonignorable (AN) Model

The core contribution of this work is the Additive Nonignorable (AN) model. Unlike MAR or HW models, which restrict attrition to depend on either the past or the present, the AN model allows the probability of dropping out to depend on both. It utilizes a logistic link function where the "willingness to respond" is an additive function of past and current variables.

Identification and Data Structure

The authors demonstrate that while the AN model is unidentified with panel data alone, the inclusion of a refreshment sample provides the necessary information to solve for missing parameters. The structure of the data, including missingness patterns across different sample groups, is formalized in the following table:

Table 1: Missing data indicators and design variables for panel and refreshment samples.

In this framework, $A_i$ represents the design variable distinguishing the original panel ($A_i=0, 1$) from the refreshment sample ($A_i=2$). The indicators $D_{i1}$ and $D_{i2}$ track missingness across waves, allowing the model to bridge the dynamics captured by the panel with the cross-sectional "truth" provided by the refreshment sample.

Estimation

Estimation is performed using a Data Augmentation (DA) algorithm, a type of Markov Chain Monte Carlo (MCMC) method. This allows for the imputation of missing values for dropouts based on patterns observed in both the refreshment sample and the remaining panel members.

Empirical Application: The Dutch Transportation Panel

The authors apply their methodology to the Dutch Transportation Panel (DTP), a longitudinal study of travel behavior in the Netherlands conducted between 1984 and 1989. The study tracked variables such as the number of trips per household and household income.

The DTP suffered from severe attrition, with 41% of the original sample dropping out after the first wave. To stabilize the study, a refreshment sample of 656 households was added in the third wave (Spring 1985). This dataset provides a real-world test case for comparing standard missing-data models against the AN model.

Key Insights and Results

The analysis reveals that the choice of model for handling attrition significantly impacts the results:

  • Bias Correction: When estimating the income elasticity of travel, a traditional MAR model yielded an estimate of 0.139. In contrast, the more robust AN model produced a significantly higher estimate of 0.213, suggesting that standard models may underestimate behavioral responses.
  • Model Rejection: Using likelihood ratio tests, the authors found that both the MAR and HW models were statistically inconsistent with the data provided by the refreshment sample. This confirms that standard assumptions were invalid for this dataset.
  • Precision: Despite the increased complexity of the AN model, the use of the refreshment sample maintained manageable standard errors, demonstrating that richer data can support more sophisticated modeling without sacrificing precision.

Conclusion and Limitations

The use of refreshment samples turns previously untestable assumptions into testable hypotheses. By comparing panel survivors to a fresh random sample, researchers can statistically validate their missing-data assumptions.

However, some limitations remain. The current approach focuses on attrition and does not address initial non-response (bias in the very first wave). Additionally, the MCMC estimation method is computationally intensive compared to simpler weighting schemes. Future research could explore the sensitivity of these results to different parametric functional forms and extend the model to handle initial selection bias.

Summary updated

Abstract

In many fields researchers wish to consider statistical models that allow for more complex relationships than can be inferred using only cross-sectional data. Panel or longitudinal data where the same units are observed repeatedly at different points in time can often provide the richer data needed for such models. Although such data allows researchers to identify more complex models than cross-sectional data, missing data problems can be more severe in panels. In particular, even units who respond in initial waves of the panel may drop out in subsequent waves, so that the subsample with complete data for all waves of the panel can be less representative of the population than the original sample. Sometimes, in the hope of mitigating the effects of attrition without losing the advantages of panel data over cross-sections, panel data sets are augmented by replacing units who have dropped out with new units randomly sampled from the original population. Following Ridder (1992), who used these replacement units to test some models for attrition, we call such additional samples refreshment samples. We explore the benefits of these samples for estimating models of attrition. We describe the manner in which the presence of refreshment samples allows the researcher to test various models for attrition in panel data, including models based on the assumption that missing data are missing at random (MAR, Rubin, 1976; Little and Rubin, 1987). The main result in the paper makes precise the extent to which refreshment samples are informative about the attrition process; a class of non-ignorable missing data models can be identified without making strong distributional or functional form assumptions if refreshment samples are available.
BibTeX
@misc{hirano1998combining,
  author = {Keisuke Hirano and Guido W. Imbens and Geert Ridder and Donald Rebin},
  title = {{Combining Panel Data Sets with Attrition and Refreshment Samples}},
  year = {1998},
  doi = {10.3386/t0230},
}
Figure