Using spatio-temporal information in weather radar data to detect and track communal bird roosts

Gustavo Pérez, Wenlong Zhao, Zezhou Cheng, Maria Carolina T. D. Belotti, Yuting Deng, Victoria F. Simons, Elske K. Tielens, Jeffrey F. Kelly, Kyle G. Horton, Subhransu Maji, Daniel SheldonbioRxiv 2022 · DOI

TL;DR

This paper introduces an AI system that uses weather radar data to automatically detect and track communal bird roosts, significantly improving upon previous methods by incorporating spatio-temporal information. This advancement enables the collection of large-scale, research-grade ecological data on declining bird populations with much less human effort.

What Problem Does This Paper Address?

Monitoring animal populations, especially migratory birds, is crucial for understanding ecological changes, but traditional methods are labor-intensive and limited in scope. Weather radar, specifically the WSR-88D network, offers a vast archive of data that can reveal bird roosting behavior, often appearing as expanding ring patterns. However, manually annotating these massive datasets to extract this information is prohibitively expensive and time-consuming. This paper addresses the challenge of efficiently and accurately extracting this valuable ecological data from decades of radar archives.

Methods

The core of the system is a Faster R-CNN architecture, a popular deep learning model for object detection. The authors customize this model by:

  • Spatio-temporal input: They augment the standard image input by concatenating multiple radar scans from consecutive time frames (up to three previous frames) and multiple radar "moments" (like reflectivity and velocity) across different elevation angles. This creates a richer input representation that captures the dynamic nature of roosts.
  • Novel adaptor layers: To handle the increased number of input channels from the spatio-temporal data, they introduce learnable "adaptor" layers that map the input to a format compatible with pre-trained networks.
  • AI-assisted screening: They developed a web interface that allows human annotators to efficiently screen the AI's predictions, significantly reducing the manual effort required for data validation.

Datasets

The study utilizes data from the US NEXRAD weather radar network, specifically focusing on 12 radar stations in the Great Lakes region. They leverage a dataset previously curated for ecological research, adapting it to the COCO format for computer vision tasks. This dataset includes annotations for communal bird roosts. The system was trained and evaluated on this dataset, and then deployed to collect 21 years of data.

Key Insights and Ideas

  • Temporal context is king: Incorporating information from previous radar scans (temporal information) dramatically improves roost detection performance. This is because it helps the model distinguish the unique, expanding movement dynamics of roosts from more static patterns like rain or ground clutter.
  • Less is more (sometimes): While adding more radar "modalities" (different types of radar measurements) and elevations didn't significantly improve performance beyond a core set (reflectivity and velocity at lower elevations), temporal information was a game-changer.
  • AI as a powerful screening tool: The AI system doesn't aim for perfect automation but acts as a highly efficient pre-screener. This AI-assisted approach drastically reduces the human effort needed to generate research-quality data, making large-scale ecological studies feasible.
  • Adaptability is key: The use of learnable adaptor layers allows the system to effectively integrate diverse radar data inputs into a standard deep learning pipeline.

Key Results

  • The deployed AI system achieved a mean average precision (mAP) of 47.0% on the test dataset, an improvement over previous work.
  • By incorporating temporal information (using three consecutive frames), the mAP increased to 54.7%.
  • The system was deployed to collect 21 years of data from 12 radar stations, resulting in the annotation of 15,628 roost signatures.
  • The AI-assisted screening process required only 1.08 seconds per radar scan on average, a massive reduction in effort compared to manual annotation.

Limitations and Future Work

  • False positives: While temporal information reduced false positives, they still exist, often due to localization errors or detections on background regions (weather, static structures). Future work could focus on more sophisticated error analysis and potentially incorporating more contextual cues that humans use.
  • False negatives: Roosts that are obscured by heavy noise (anomalous propagation, ground clutter), lack a distinct ring shape, or are too close to other roosts can be missed.
  • Rain discrimination: Distinguishing rain from roosts remains a challenge, and future research might explore multi-head models for joint roost detection and rain segmentation.
  • Broader applications: The authors suggest that this framework can be extended to detect and track other biological phenomena in radar data, such as bat roosts and mayfly hatches, opening up new avenues for radar aeroecology research. They also propose exploring finer-grained distinctions between different swallow species and extending the models to radar networks outside the US.

Summary updated

Abstract

Abstract The exodus of swallows from communal nighttime roosts is often visible as an expanding ring-shaped pattern in weather radar data. The WSR-88D network operated by the National Weather Service archives more than 25 years of data across 143 stations in the contiguous US. However, access to information about the roosting behavior of swallows is limited by the cost of manual annotation of these scans. We develop an AI system to detect and track swallow roosts in weather radar data. Our model is based on the Faster R-CNN architecture and is customized to incorporate multiple spatial and temporal channels in volumetric radar scans using novel adaptor layers. We systematically study the impact of network architecture and input representation for this task. We incorporate our detection outputs into an AI-assisted system with an interface for human screening to collect research-grade data about roosting behavior. We deploy the system to collect information from 12 radar stations in the Great Lakes region of the US spanning 21 years. The addition of temporal information improves roost detection performance from 47.0% mean average precision to 54.7%. Temporal information helps the model recognize the expanding pattern of roosts and filter false positives due to rain and static structures. Our system allowed the annotation of 15,628 roost signatures with 64,620 single-frame detections in 612,786 radar scans with 183.6 total hours of human screening, or 1.08 seconds per radar scan. Our AI-assisted system provides research-quality roost data with far less human effort than manual annotation of radar scans. The data contains critical information about the phenology and population trends of swallows and martins, a declining group of aerial insectivores. Our successful deployment to collect historical data for 8% of the radar stations in the contiguous US lays the groundwork for continentscale analysis of swallow roosts, and provides a starting point for analysis of other family-specific phenomena in weather radar, such as bat roosts and mayfly hatches.
BibTeX
@misc{perez2022using,
  author = {Gustavo Pérez and Wenlong Zhao and Zezhou Cheng and Maria Carolina T. D. Belotti and Yuting Deng and Victoria F. Simons and Elske K. Tielens and Jeffrey F. Kelly and Kyle G. Horton and Subhransu Maji and Daniel Sheldon},
  title = {{Using spatio-temporal information in weather radar data to detect and track communal bird roosts}},
  year = {2022},
  doi = {10.1101/2022.10.28.513761},
  howpublished = {bioRxiv},
}
Figure