Abstract
With panel data, important issues can be resolved that cannot be addressed with cross-sectional data. A major drawback is that panel data suffer from more severe missing data problems. Adding a sample consisting of new units randomly drawn from the original population as replacements for units who have dropped out of the panel—a so-called refreshment sample—can be helpful in mitigating the effects of attrition, both by allowing for estimation of richer models and by making estimation of conventional models more precise. In this paper, we develop a family of models that incorporate refreshment samples, and we demonstrate in an application to a Dutch data set on travel behavior that such models can lead to substantially different results than models that assume that the missing data process is ignorable or conventional econometric models for panel data with attrition.
Introduction
In economics and social sciences, panel data is highly valued for its ability to follow the same individuals over time, allowing researchers to control for unobserved individual characteristics. However, longitudinal studies frequently suffer from "attrition"—the tendency for participants to drop out over successive waves. If this dropout process is non-random (e.g., high-income individuals being too busy to respond), the remaining sample becomes biased, leading to incorrect inferences.
Traditionally, researchers have relied on two primary, often untestable, assumptions to address missing data:
- Missing at Random (MAR): Assumes that dropouts depend only on data collected in previous waves.
- Hausman-Wise (HW): Assumes that dropouts depend on current, unobserved values.
This paper introduces a more flexible approach using "refreshment samples"—separate, random samples taken at the same time as later panel waves—to detect and correct for non-random attrition.
Methodology: The Additive Nonignorable (AN) Model
The core contribution of this work is the Additive Nonignorable (AN) model. Unlike MAR or HW models, which restrict attrition to depend on either the past or the present, the AN model allows the probability of dropping out to depend on both. It utilizes a logistic link function where the "willingness to respond" is an additive function of past and current variables.
Identification and Data Structure
The authors demonstrate that while the AN model is unidentified with panel data alone, the inclusion of a refreshment sample provides the necessary information to solve for missing parameters. The structure of the data, including missingness patterns across different sample groups, is formalized in the following table:
In this framework, $A_i$ represents the design variable distinguishing the original panel ($A_i=0, 1$) from the refreshment sample ($A_i=2$). The indicators $D_{i1}$ and $D_{i2}$ track missingness across waves, allowing the model to bridge the dynamics captured by the panel with the cross-sectional "truth" provided by the refreshment sample.
Estimation
Estimation is performed using a Data Augmentation (DA) algorithm, a type of Markov Chain Monte Carlo (MCMC) method. This allows for the imputation of missing values for dropouts based on patterns observed in both the refreshment sample and the remaining panel members.
Empirical Application: The Dutch Transportation Panel
The authors apply their methodology to the Dutch Transportation Panel (DTP), a longitudinal study of travel behavior in the Netherlands conducted between 1984 and 1989. The study tracked variables such as the number of trips per household and household income.
The DTP suffered from severe attrition, with 41% of the original sample dropping out after the first wave. To stabilize the study, a refreshment sample of 656 households was added in the third wave (Spring 1985). This dataset provides a real-world test case for comparing standard missing-data models against the AN model.
Key Insights and Results
The analysis reveals that the choice of model for handling attrition significantly impacts the results:
- Bias Correction: When estimating the income elasticity of travel, a traditional MAR model yielded an estimate of 0.139. In contrast, the more robust AN model produced a significantly higher estimate of 0.213, suggesting that standard models may underestimate behavioral responses.
- Model Rejection: Using likelihood ratio tests, the authors found that both the MAR and HW models were statistically inconsistent with the data provided by the refreshment sample. This confirms that standard assumptions were invalid for this dataset.
- Precision: Despite the increased complexity of the AN model, the use of the refreshment sample maintained manageable standard errors, demonstrating that richer data can support more sophisticated modeling without sacrificing precision.
Conclusion and Limitations
The use of refreshment samples turns previously untestable assumptions into testable hypotheses. By comparing panel survivors to a fresh random sample, researchers can statistically validate their missing-data assumptions.
However, some limitations remain. The current approach focuses on attrition and does not address initial non-response (bias in the very first wave). Additionally, the MCMC estimation method is computationally intensive compared to simpler weighting schemes. Future research could explore the sensitivity of these results to different parametric functional forms and extend the model to handle initial selection bias.
Summary updated