When we track a group of individuals over time, we naturally assume that changes in our data reflect the natural trajectory of those individuals. If a cohort's average health score remains stable over five years, we conclude that the population is holding steady. If a machine learning model maintains high predictive accuracy over a multi-month study, we trust its robustness.
This conclusion is often an illusion.
Longitudinal data is continuously shaped by a hidden, non-random filtration process: attrition bias. Over time, a cohort changes not just because its individuals change, but because who stays in the cohort changes. When the probability of dropping out is correlated with the very variables we want to measure, the remaining sample becomes a funhouse mirror of the original population.
To build reliable models and draw accurate causal inferences, we must understand the distinct mechanisms that drive this filtration.
The Biological Filter: Survival Bias and Hardy Phenotypes
The most absolute filter in longitudinal research is mortality or severe clinical deterioration. When the most vulnerable individuals exit a study because they die or become too ill to participate, the remaining cohort becomes systematically healthier over time.
In early-childhood health research, Jade Benjamin‐Chung et al. demonstrate that longitudinal studies of growth faltering are highly susceptible to survivor bias. Because the most vulnerable children face early mortality, they are excluded from long-term analysis, causing researchers to underestimate the true severity of growth deficits in high-risk populations.
A similar phenomenon occurs in studies of aggressive diseases. Maureen Parkinson et al. observe that in longitudinal studies of high-grade glioma patients, cognitive scores often appear stable over time. However, this stability is an artifact of attrition: patients experiencing rapid neurological decline drop out or pass away, leaving behind a subset of responders whose stable trajectories mask the broader cohort's decline.
Initial Cohort ───► [ High Vulnerability ] ───► Early Mortality / Dropout (Lost to Data)
───► [ Low Vulnerability ] ───► Survives to End (Artificially Stable Sample)
This biological filtering can even occur before a study officially begins, a challenge known as left truncation. As Andrew Ying notes, left truncation occurs when individuals are only observed if they have survived to a certain age or state to enter the study. For example, Corinne Pettigrew et al. point out that requiring participants to be cognitively healthy at baseline inadvertently selects for a "hardy phenotype." These individuals have already bypassed early-onset genetic risks, which can attenuate the observed relationship between genetic risk factors and subsequent cognitive decline.
The Behavioral Filter: The Experience Feedback Loop
Attrition is also driven by how participants subjectively experience their condition or treatment. This creates a feedback loop where the data points we lose are directly caused by the outcomes we are trying to measure.
As Peng Wu et al. explain, the availability of long-term data is rarely random; it is highly conditional on a participant's short-term response to a treatment or platform. If a user or patient has a negative immediate experience, they are far more likely to disengage.
This behavioral dropout warps clinical evaluations in two opposite directions:
- Overestimating Tolerability: In a study on prostate cancer therapies, Jasmine Lim et al. note that patients experiencing severe side effects are the most likely to withdraw. This selective attrition skews the remaining data, painting an overly optimistic picture of the treatment's tolerability and health-related quality of life.
- Overestimating Severity: Conversely, if a study relies on patients actively returning to a healthcare system, the filter reverses. Sherif Shoucri et al. show that in retrospective COVID-19 tracking, individuals who fully recover are less likely to seek further care or return for follow-ups. The remaining clinical sample is disproportionately composed of those with persistent symptoms, leading to an overestimation of long-term complications.
The Compliance and Structural Filter
A third mechanism is rooted in the social, economic, and operational realities of long-term participation. Remaining in a study requires time, trust, and resources, meaning that attrition often follows sharp socio-demographic lines.
Katja Radon et al. find that longitudinal cohorts frequently suffer from demographic attrition, where younger individuals and those from lower socio-economic backgrounds are disproportionately lost over time. When tracking childhood health, Zhengcun Pei et al. demonstrate that this selective loss of high-risk or lower-income families biases risk models and can lead to a significant underestimation of the prevalence of conditions like childhood obesity.
Even well-intentioned data processing choices can inadvertently introduce this structural bias. In digital health studies that use passive smartphone sensors, researchers must often filter out users with incomplete data. However, Joy He-Yueya et al. caution that rigorous data filtering for modeling can drastically reduce sample sizes, leaving behind a highly compliant, unrepresentative subset of participants who maintain consistent device engagement, while dropping those with chaotic routines—the very individuals the models may need to predict.
Why It Breaks Machine Learning: Dynamic Selection Bias
For machine learning and prognostic modeling, non-random attrition introduces a insidious problem: the data generating process changes underneath the model.
Consider a model designed to predict patient outcomes over the course of an intensive care unit (ICU) stay. As Christopher Meiring et al. demonstrate, as time progresses, patients with straightforward outcomes (either rapid recovery or rapid death) leave the cohort. The remaining population evolves into a highly complex, volatile, and concentrated group of patients who are inherently harder to predict.
If an algorithm is trained on the baseline population, its performance will degrade over time because it fails to account for this dynamic selection bias. The model assumes it is evaluating the same distribution of patients, but the attrition process has fundamentally altered the underlying feature space.
To build truly generalizable longitudinal models, researchers cannot treat missing data as a simple statistical inconvenience. Attrition is a behavior in itself—a signal that must be modeled, robustly adjusted for, or explicitly accounted for through double-robust estimation frameworks to prevent our models from learning only the patterns of the survivors.
Key Foundational Work
Combining Panel Data Sets with Attrition and Refreshment Samples (1998) — contributed 3 concept elements to this direction.