TL;DR
This paper explores how Unoccupied Aerial Systems (UAS) equipped with machine learning can improve waterfowl surveys. By analyzing factors like vegetation, weather, and image resolution, they developed a method to correct for detection biases, significantly reducing counting errors and paving the way for more accurate wildlife monitoring.
What Problem Does This Paper Address?
Traditional methods for surveying waterfowl populations, like ground-based observations and manned aircraft surveys, face significant challenges. Ground surveys can be hindered by vegetation, while manned aerial surveys are expensive, potentially hazardous, and can disturb the birds. Furthermore, the lack of standardization across different regions makes it difficult to share data and learn from conservation efforts. While Unoccupied Aerial Systems (UAS) offer a promising alternative due to their cost-effectiveness, safety, and potential for high-resolution imagery, accurately counting waterfowl from UAS imagery is still hampered by biases. These biases include:
- Availability Bias: Waterfowl present in the survey area might not be visible or accessible for detection due to factors like vegetation cover or their behavior.
- Perception Bias: Even when visible, birds might be missed by automated detection systems (false negatives) or non-bird objects might be misidentified as birds (false positives).
This paper aims to quantify these biases in UAS-based surveys and develop a method to correct for them, ultimately improving the accuracy of waterfowl abundance estimates.
Methods
The researchers employed a multi-faceted approach combining UAS data collection, expert labeling, and statistical modeling:
- UAS Data Collection: They used a DJI Mavic Pro 2 drone with proprietary software for automated flight path planning. Imagery was collected at various ground sampling distances (GSDs) – ranging from 0.38 to 2.29 cm/pixel, corresponding to altitudes of 15 to 90 meters – across different vegetation cover types and under varying sky conditions.
- Decoy-Based Availability Assessment: To specifically evaluate availability bias, they deployed decoy flocks of six common waterfowl species. This allowed them to control for the presence of birds and isolate factors affecting their visibility.
- Live Bird Perception Assessment: For perception bias, they conducted surveys over actual waterfowl populations.
- Machine Learning Detection: A modified YOLONAS object detection algorithm was used to automatically identify and classify waterfowl in the aerial images.
- Statistical Modeling: Generalized linear mixed models with Bernoulli distributions were used to quantify the influence of environmental factors (vegetation cover type, sky condition, GSD) and bird characteristics (species, sex) on both availability and perception biases.
- Bias Correction: A modified Horvitz–Thompson (H–T) estimator was applied to incorporate the estimated detection probabilities (both availability and perception biases, including false positives) to correct the raw algorithm counts and provide more accurate abundance estimates.
Datasets
- UAS Imagery: The study utilized imagery collected from DJI Mavic Pro 2 drones over ten Missouri Department of Conservation intensively managed wetland Conservation Areas. This imagery was collected between October 2021 and March 2022.
- Decoy Data: Imagery containing 180 decoys per image was used to assess availability bias. These images were labeled by trained experts.
- Live Bird Data: Imagery of live waterfowl was used to assess perception bias. A total of 404 images containing 105,392 individual birds were analyzed for this purpose.
- Benchmark Data: While not explicitly a new benchmark dataset in the traditional sense, the study generated a valuable dataset of labeled waterfowl images that can be used for future algorithm development and validation in similar environments.
Key Insights and Ideas
- Availability is Key: The study strongly emphasizes that waterfowl availability for detection is significantly influenced by environmental factors, particularly vegetation complexity and GSD. This means that when and how you fly the drone matters as much as the detection algorithm itself.
- Vegetation is a Double-Edged Sword: While complex vegetation can hide birds (availability bias), it can also create features that confuse algorithms, leading to false positives (perception bias).
- Sky Conditions are Less Critical Than Expected: Surprisingly, sky conditions (cloudy vs. sunny) had a less pronounced effect on waterfowl availability and detection compared to vegetation and GSD, contrary to some previous assumptions.
- The Power of Correction: The modified Horvitz–Thompson estimator proved effective in correcting for both availability and perception biases, significantly reducing the overall error in waterfowl counts. This highlights the importance of not just detecting birds, but also accounting for the inherent uncertainties in the detection process.
- Species and Sex Matter for Availability: The study found that species and sex influenced how available birds were for detection, likely due to differences in plumage and size, which interact with environmental factors.
Key Results
- Availability Bias:
- Waterfowl availability was significantly reduced in more complex and taller vegetation cover types. Open water offered the highest availability (99.78%), while forested areas had the lowest (39.90%).
- Increasing GSD (higher altitude) generally decreased waterfowl availability.
- Species and sex influenced availability, with males generally being more available than females. Mallards and northern shovelers had higher availability than gadwalls.
- Perception Bias:
- The probability of correctly detecting a bird was not significantly influenced by vegetation cover type, GSD, or sky condition.
- False-positive detections were more likely in vegetation cover types with features similar in size and shape to birds, particularly lotus.
- The combination of lotus vegetation, cloudy skies, and a GSD of 60m resulted in the highest rate of false positives.
- Corrected Counts:
- The modified H–T estimator reduced the mean absolute error of waterfowl counts by half, from 6.70% for raw algorithm outputs to 3.35% for corrected counts.
- The correction factors varied, ranging from 1.0 to 6.5 depending on environmental conditions and waterfowl characteristics.
Limitations and Future Work
- Annotation Limitations: The study acknowledges two sources of error in their annotation process:
- Only labeling "whole birds" meant that partial birds at image edges, if detected by the algorithm, were considered false positives.
- Cropping and stitching images could lead to duplicate labels for the same bird if bounding boxes didn't perfectly align. These errors were estimated to account for a small percentage of false positives and detection errors.
- Species/Sex Specificity: While the study evaluated species and sex for availability, the perception bias analysis focused on overall detection of birds, not specific species or sexes. This was due to limitations in labeling birds to that level of detail. Future work could explore species- and sex-specific detection probabilities.
- Generalizability: The study was conducted in specific wetland areas in Missouri. Further research is needed to evaluate the performance of these methods in different geographical regions and wetland types.
- Algorithm Training: The authors suggest that future algorithms could benefit from more robust training on diverse scenarios, especially those involving challenging cover types like lotus, to further reduce false positives.
- Landscape-Level Estimates: While the study focused on per-image corrections, future research could explore how to scale these corrections to landscape or survey unit levels for more comprehensive abundance estimates.
Summary updated