Abstract
The US weather radar archive holds detailed information about biological phenomena in the atmosphere over the last 20 years. Communally roosting birds congregate in large numbers at nighttime roosting locations, and their morning exodus from the roost is often visible as a distinctive pattern in radar images. This paper describes a machine learning system to detect and track roost signatures in weather radar data. A significant challenge is that labels were collected opportunistically from previous research studies and there are systematic differences in labeling style. We contribute a latent variable model and EM algorithm to learn a detection model together with models of labeling styles for individual annotators. By properly accounting for these variations we learn a significantly more accurate detector. The resulting system detects previously unknown roosting locations and provides comprehensive spatio-temporal data about roosts across the US. This data will provide biologists important information about the poorly understood phenomena of broad-scale habitat use and movements of communally roosting birds during the non-breeding season.
Introduction: The Radar as a Biological Sensor
The US NEXRAD weather radar network is an accidental goldmine for biologists. While designed to track storms, its 143 stations are sensitive enough to detect "biological targets"—birds, bats, and insects. Specifically, communal roosting birds like Tree Swallows create a distinctive "expanding ring" pattern on radar as hundreds of thousands of birds take flight simultaneously at sunrise.
However, manually analyzing two decades of data across the entire continent is impossible. Previous attempts at automation were stymied by a "labeling style" problem: different human experts have different ways of marking roosts (some draw tight circles, others loose ones). This inconsistent supervision makes it incredibly difficult to train and evaluate standard computer vision models, as the "ground truth" itself is subjective.
Methods
The researchers developed a multi-stage pipeline to transform raw radar data into biological insights, addressing both the technical challenges of radar processing and the human challenges of inconsistent labeling.
1. Single-Frame Detection
The system adapts Faster R-CNN, a standard object detection framework, to radar data. Since the model expects three-channel RGB images, the authors selected the three most discriminative radar products: reflectivity at two different elevation angles and radial velocity.
2. Handling Labeling Bias
To solve the inconsistent labeling problem, the authors developed a latent variable model and a Variational EM (Expectation-Maximization) algorithm. This approach treats the "true" roost size as a hidden variable and learns a "user model" for each annotator. By estimating how much each user tends to over- or under-scale their labels, the system can "normalize" the training data, leading to a significantly more accurate detector.
3. Tracking and Rescoring
Detections are linked across time using a "detect-then-track" approach with a greedy heuristic, followed by smoothing with a Kalman filter. To eliminate false positives, detections are cross-referenced with the U.S. Wind Turbine Database and filtered using dual-polarization radar data to distinguish birds from rain.
Datasets
The study utilized several key data sources:
- Primary Labeled Set: 88,972 radar scans from seven stations in the Eastern US, containing 63,691 manually labeled roosts.
- NEXRAD Archive: A massive "in the wild" dataset of 419,000 scans from 86 radar stations collected between October 2013 and March 2014.
- Auxiliary Data: The National Land Cover Database (NLCD) for habitat analysis and the U.S. Wind Turbine Database for noise reduction.
Key Results
The inclusion of the user model for labeling bias proved critical. Overall Mean Average Precision (MAP) improved from 41.0% to 45.5%. In cases with high labeling variance, such as the KDOX station, the improvement was dramatic, with MAP jumping from 9.1% to 50.2%.
In a "blind" test of the top 521 tracks identified by the system, 93.5% were correctly identified as bird roosts. Biologically, the system successfully mapped the 2013-2014 fall migration, revealing movement patterns along the eastern seaboard and Mississippi Valley and calculating an average dispersal speed of approximately 6.61 m/s.
Conclusion and Future Work
This system demonstrates that accounting for human subjectivity is essential when training models on expert-labeled biological data. While currently optimized for the unique "ring" signature of Tree Swallows, future work could extend the system to other species and conduct a full 20-year historical analysis. Such long-term data would be invaluable for studying the impact of climate change on bird populations and habitat use.
Summary updated