Artificial intelligence for automated detection of large mammals creates path to upscale drone surveys

Javier Lenzi, Andrew F. Barnas, Abdelrahman A. ElSaid, Travis Desell, Robert F. Rockwell, Susan N. Ellis‐FelegeScientific 2023 · DOI

TL;DR

This paper introduces a Faster R-CNN model for automated detection and counting of caribou from drone imagery, achieving high accuracy (80%) and precision (90%). This AI-driven approach offers a scalable and efficient alternative to manual surveys, significantly reducing processing time and improving consistency.

What Problem Does This Paper Address?

Wildlife research and management increasingly rely on drone imagery for surveys. However, processing the vast amounts of data generated by these drones is a significant bottleneck. Manual counting of animals from aerial images is time-consuming, prone to errors, and difficult to scale for large populations or vast geographical areas. This challenge is particularly acute in remote and logistically difficult environments like the Arctic, where caribou populations are of conservation concern. Existing methods struggle with efficiency and accuracy, necessitating the development of automated solutions.

Methods

The researchers developed a methodology for training a convolutional neural network (CNN) on large-scale mosaic drone imagery. Specifically, they employed a Faster R-CNN (Faster Region-based Convolutional Neural Network) model, a well-established deep learning architecture for object detection.

Key methodological aspects include:

  • Data Preparation: Large mosaic images were tiled into smaller, manageable pieces (1000x1000 pixels) for training. Overlapping tiles were used to mitigate issues with animals being split across tile boundaries.
  • Annotation and Labeling: Images were annotated with three labels: "adult caribou," "calf caribou," and "ghost caribou." The "ghost caribou" label was introduced to account for blurred individuals resulting from movement during photogrammetry processing, a common challenge in drone imagery.
  • Training and Validation: The Faster R-CNN model was trained using a dataset of annotated tiles. The model's performance was then evaluated against an experienced human observer, considered the ground truth.
  • Comparison with Human Observers: The study compared the AI model's performance not only with an expert but also with a group of "naïve" observers to assess its effectiveness in a more realistic, multi-observer scenario.

Datasets

The study utilized drone imagery collected on July 18, 2016, in the braided delta of the Mast River in Wapusk National Park, Manitoba, Canada.

  • Imagery Source: Images were captured by a Trimble UX5 fixed-wing drone equipped with a 16.1 MP optical camera.
  • Altitudes: Imagery was collected at two altitudes: 75 m and 120 m above ground level (AGL).
  • Mosaic Creation: Individual images were stitched together using Pix4D v. 3.1.22 to create four georeferenced orthomosaic TIFF images.
  • Training/Testing Split: A total of 2,148 tiles were generated. This dataset was split into a training set (1607 tiles, 75%) and a testing set (541 tiles, 25%). The testing set included negative examples (tiles without caribou).

Key Insights and Ideas

  • The "Ghost" Caribou Challenge: The introduction of the "ghost caribou" label is a novel approach to handle a common artifact in drone photogrammetry, where moving animals appear blurred. This demonstrates an understanding of the practical challenges in drone data processing.
  • Altitude Matters (Slightly): The model performed slightly better at higher altitudes (120 m AGL) in terms of accuracy, suggesting that while the model is robust to altitude changes, there can be subtle performance differences.
  • AI Outperforms Naïve Observers: The Faster R-CNN model generally performed better than the group of naïve observers, particularly in detecting adults and calves. This highlights the potential for AI to reduce variability introduced by human interpretation.
  • Consistency is Key for Training: The study emphasizes the critical need for consistent annotations from human observers when training AI models. Inconsistencies can undermine the accuracy of even well-trained algorithms.
  • Scalability is the Goal: The overarching idea is to leverage AI to overcome the scalability limitations of manual drone data analysis, enabling more comprehensive and frequent wildlife surveys.

Key Results

  • Overall Performance: The Faster R-CNN model achieved an overall accuracy of 80%, precision of 90%, and recall of 88%.
  • Altitude Performance: Accuracies were higher at 120 m AGL (80-88%) compared to 75 m AGL (75-76%).
  • Category Performance: The model was more efficient at detecting adults and calves than ghosts.
  • Correlation with Expert: Detections from the Faster-RCNN model were highly correlated with the experienced observer (Pearson correlation coefficients ranging from 0.96 to 0.99).
  • Time Efficiency: While training the model took approximately one week, processing the testing set took only 19.8 minutes, a significant improvement over the hours required by human observers.
  • Misclassification Analysis: The model showed a tendency to overestimate detections compared to the experienced observer. The most common misclassification for both the model and naïve observers was "adult as ghost."

Limitations and Future Work

  • Ghost Detection Challenges: The "ghost caribou" category, while useful, remained challenging for both the AI model and human observers, indicating an area for further improvement.
  • Object Misclassification: The Faster R-CNN model occasionally misclassified non-caribou objects (e.g., rocks, trees) as caribou, a limitation not observed with human observers.
  • Generalization: The authors acknowledge the need for generalization of automated methods to diverse platforms, airspace restrictions, and sensor capabilities for large-scale studies.
  • Observer Consistency: Improving the consistency of human annotations is crucial for accurate model training and validation, especially for large-scale projects.
  • Double Counting: The paper touches upon the issue of double counting individuals, a persistent challenge in mosaicked imagery, and suggests exploring alternative methods like object tracking or multi-drone coordination.
  • Future Directions: The authors suggest exploring more advanced techniques like hierarchical models and modified Horvitz-Thompson methods to account for imperfect detection and improve abundance estimation. They also highlight the potential for training neural networks with data from varying ground sampling distances and different environmental conditions.

Summary updated

Abstract

Imagery from drones is becoming common in wildlife research and management, but processing data efficiently remains a challenge. We developed a methodology for training a convolutional neural network model on large-scale mosaic imagery to detect and count caribou (Rangifer tarandus), compare model performance with an experienced observer and a group of naïve observers, and discuss the use of aerial imagery and automated methods for large mammal surveys. Combining images taken at 75 m and 120 m above ground level, a faster region-based convolutional neural network (Faster-RCNN) model was trained in using annotated imagery with the labels: "adult caribou", "calf caribou", and "ghost caribou" (animals moving between images, producing blurring individuals during the photogrammetry processing). Accuracy, precision, and recall of the model were 80%, 90%, and 88%, respectively. Detections between the model and experienced observer were highly correlated (Pearson: 0.96-0.99, P value < 0.05). The model was generally more effective in detecting adults, calves, and ghosts than naïve observers at both altitudes. We also discuss the need to improve consistency of observers' annotations if manual review will be used to train models accurately. Generalization of automated methods for large mammal detections will be necessary for large-scale studies with diverse platforms, airspace restrictions, and sensor capabilities.
BibTeX
@inproceedings{lenzi2023artificial,
  author = {Javier Lenzi and Andrew F. Barnas and Abdelrahman A. ElSaid and Travis Desell and Robert F. Rockwell and Susan N. Ellis‐Felege},
  title = {{Artificial intelligence for automated detection of large mammals creates path to upscale drone surveys}},
  booktitle = {Scientific},
  year = {2023},
  doi = {10.1038/s41598-023-28240-9},
}
Figure