Statistical Inference on Tree Swallow Migrations with Random Forests

Tim Coleman, Lucas Mentch, Daniel Fink, Frank La Sorte, Giles Hooker, Wesley M. Hochachka, David W. WinklerJournal of the Royal Statistical Society Series C (Applied Statistics) 2020 · DOI · arXiv

Abstract

Bird species’ migratory patterns have typically been studied through individual observations and historical records. In recent years, the eBird citizen science project, which solicits observations from thousands of bird watchers around the world, has opened the door for a data-driven approach to understanding large-scale geographical movements. Here, we focus on the North American tree swallow (Tachycineta bicolor) occurrence patterns throughout the eastern USA. Migratory departure dates for this species are widely believed by both ornithologists and casual observers to vary substantially across years, but the reasons for this are largely unknown. In this work, we present evidence that maximum daily temperature is predictive of tree swallow occurrence. Because it is generally understood that species occurrence is a function of many complex, high-order interactions between ecological covariates, we utilize the flexible modelling approach offered by random forests. Making use of recent asymptotic results, we provide formal hypothesis tests for predictive significance of various covariates and also develop and implement a permutation-based approach for formally assessing interannual variations by treating the prediction surfaces generated by random forests as functional data. Each of these tests suggests that maximum daily temperature is important in predicting migration patterns.

The Challenge of Facultative Migration

Ornithologists have long observed that Tree Swallows are "facultative migrants"—they do not follow a rigid, calendar-based flight schedule but instead respond dynamically to local environmental conditions. This behavior was particularly evident in 2008 and 2009, when observers noted birds lingering in the northern United States much longer than expected before suddenly departing.

While these anomalies were noted qualitatively, proving they were statistically significant and identifying their drivers is difficult. Traditional parametric models, such as linear regression, often fail to capture the complex, high-order interactions between weather, land cover, and elevation that define ecological data. While Random Forests (RF) excel at capturing these complexities, they have historically been viewed as "black-box" models, lacking the tools for formal hypothesis testing and p-value generation required for rigorous scientific inference.

Bridging Machine Learning and Statistical Inference

To address this gap, this research employs two innovative strategies to extract statistical truth from Random Forest models:

  1. Functional Permutation Tests: To compare migration patterns between year groups (e.g., the anomalous 2008-09 vs. the baseline 2010-13), the authors treat the RF's predicted occurrence over the entire season as a "prediction surface" or functional data. To manage computational costs, they developed a "mini-ensemble" approach, permuting groups of trees to create an exchangeable sequence of functions for testing.
  2. RFs as U-Statistics: For localized testing, the study utilizes recent asymptotic results that frame subsampled Random Forests as infinite-order U-statistics. This allows for the calculation of a formal covariance matrix and the use of Chi-squared tests to determine if specific variables, such as temperature anomalies, significantly contribute to the model’s predictive power at specific geographic coordinates.

Datasets

The study integrated several high-resolution datasets:

  • eBird: 173,002 unique observation checklists from the Mid-Atlantic/New England coast (2008–2013).
  • DayMet: Daily meteorological data, including maximum and minimum temperatures.
  • MODIS & UMD Land Cover: Remotely sensed habitat classification in 3km x 3km grids.
  • ASTER Global DEM: High-resolution elevation data.

Identifying Temperature as a Key Driver

The models confirmed that Tree Swallows stayed significantly longer in northern regions during 2008 and 2009. The functional permutation test yielded a p-value of 0.002, providing strong evidence that the 2008-2009 migration was distinct.

Predicted occurrence of Tree Swallows by day of year, comparing the anomalous 2008-2009 period (red) against the 2010-2013 baseline. The higher occurrence between days 220 and 300 illustrates the "lingering" effect.

The researchers argue that maximum daily temperature acts as a critical proxy for food availability. Tree Swallows rely on flying insects, which become inactive or die off during "cold snaps" (days below 18.5°C). Localized testing revealed a "transition zone" in the Mid-Atlantic where temperature fluctuations have the most dramatic impact; in the far north, it is consistently too cold by late autumn, while the far south remains warm enough. It is in these middle latitudes where weather variability dictates whether a swallow stays or goes.

Key Results and Discussion

  • Predictive Superiority: In 3-fold cross-validation, Random Forests outperformed Neural Networks (ANN), K-Nearest Neighbors (KNN), and Linear Discriminant Analysis (LDA).
  • Local Impact: In four out of six specific wildlife refuges tested, the "temperature anomaly" variable was a significant predictor of occurrence ($p < 0.001$ in several cases), validating the importance of local weather over simple seasonal averages.

While the study provides strong evidence that temperature is predictive, the authors note the distinction between correlation and causation. Unobserved variables, such as direct insect density measurements, remain a challenge for true causal inference. Additionally, while the study controlled for observer effort, citizen science data remains spatially biased toward accessible areas. Future work may apply these functional inference techniques to other migratory insectivores to determine if temperature-driven lingering is a widespread response to a changing climate.

Summary updated

Abstract

Summary Bird species’ migratory patterns have typically been studied through individual observations and historical records. In recent years, the eBird citizen science project, which solicits observations from thousands of bird watchers around the world, has opened the door for a data-driven approach to understanding the large-scale geographical movements. Here, we focus on the North American tree swallow (Tachycineta bicolor) occurrence patterns throughout the eastern USA. Migratory departure dates for this species are widely believed by both ornithologists and casual observers to vary substantially across years, but the reasons for this are largely unknown. In this work, we present evidence that maximum daily temperature is predictive of tree swallow occurrence. Because it is generally understood that species occurrence is a function of many complex, high order interactions between ecological covariates, we utilize the flexible modelling approach that is offered by random forests. Making use of recent asymptotic results, we provide formal hypothesis tests for predictive significance of various covariates and also develop and implement a permutation-based approach for formally assessing interannual variations by treating the prediction surfaces that are generated by random forests as functional data. Each of these tests suggest that maximum daily temperature is important in predicting migration patterns.
BibTeX
@article{coleman2020statistical,
  author = {Tim Coleman and Lucas Mentch and Daniel Fink and Frank La Sorte and Giles Hooker and Wesley M. Hochachka and David W. Winkler},
  title = {{Statistical Inference on Tree Swallow Migrations with Random Forests}},
  journal = {Journal of the Royal Statistical Society Series C (Applied Statistics)},
  year = {2020},
  doi = {10.1111/rssc.12416},
}
Figure