Assessing the potential for deep learning and computer vision to identify bumble bee species from images

Brian J. Spiesman, Claudio Gratton, Richard G. Hatfield, William H. Hsu, Sarina Jepsen, Brian McCornack, Krushi Patel, Guanghui WangScientific 2021 · DOI

Abstract

Pollinators are undergoing a global decline. Although vital to pollinator conservation and ecological research, species-level identification is expensive, time consuming, and requires specialized taxonomic training. However, deep learning and computer vision are providing ways to open this methodological bottleneck through automated identification from images. Focusing on bumble bees, we compare four convolutional neural network classification models to evaluate prediction speed, accuracy, and the potential of this technology for automated bee identification. We gathered over 89,000 images of bumble bees, representing 36 species in North America, to train the ResNet, Wide ResNet, InceptionV3, and MnasNet models. Among these models, InceptionV3 presented a good balance of accuracy (91.6%) and average speed (3.34 ms). Species-level error rates were generally smaller for species represented by more training images. However, error rates also depended on the level of morphological variability among individuals within a species and similarity to other species. Continued development of this technology for automatic species identification and monitoring has the potential to be transformative for the fields of ecology and conservation. To this end, we present BeeMachine, a web application that allows anyone to use our classification model to identify bumble bees in their own images.

Introduction

Pollinators are in global decline, making large-scale monitoring essential for conservation. However, identifying bumble bees at the species level is notoriously difficult. Traditional methods require specialized taxonomic expertise and often involve "lethal sampling"—collecting, pinning, and examining specimens under a microscope to see minute features like genitalia or facial structures.

While citizen science platforms like iNaturalist have increased the volume of data, human-sourced identifications for bees are often inaccurate, typically agreeing with experts only 53–73% of the time. This creates a massive data processing bottleneck. This research explores whether computer vision can provide a fast, accurate, and non-invasive alternative to manual identification.

Methodology

The researchers employed a transfer learning approach, taking four established convolutional neural network (CNN) architectures pre-trained on the ImageNet database and fine-tuning them on a custom bumble bee dataset. The models compared were:

  • ResNet-101 & Wide-ResNet-101: Known for "skip connections" that allow for very deep or wide architectures without losing signal.
  • InceptionV3: Optimized for computational efficiency by using factorized convolutions.
  • MnasNet-A1: A lightweight model designed specifically for mobile devices.

The workflow included an automated preprocessing step where an object detection algorithm "tightly cropped" the bee in each image to remove background noise. They also used data augmentation (rotation, shearing, and flipping) to improve model generalization across different lighting and orientations.

Datasets and Morphological Challenges

The study utilized a massive dataset of 89,776 images representing 36 species of North American bumble bees. Images were aggregated from Bumble Bee Watch, iNaturalist, and BugGuide. Only "verified" or "research grade" images were included. Species with fewer than 150 images were excluded, and the number of images per species was capped at 10,000 to prevent bias.

A high-quality field image showing the side profile and diagnostic features of a bumble bee.

Accuracy is not solely dependent on the number of photos; it is also heavily influenced by "intraspecific variability." For example, B. fraternus has a very stable, distinctive appearance and was identified accurately despite a small sample size. Conversely, highly variable species often confused with others lead to higher error rates.

Example of a bumble bee with distinct orange abdominal banding, illustrating the morphological diversity the models must learn.

Key Results

  • Winner: InceptionV3 emerged as the most practical model, offering a "sweet spot" of 91.6% accuracy and a lightning-fast inference speed of 3.34 ms per image.
  • Highest Accuracy: Wide-ResNet-101 was slightly more accurate (91.7%) but significantly slower (5.46 ms).
  • Top-5 Performance: All models achieved over 98% Top-5 accuracy, meaning the correct species was almost always in the model's "shortlist."
  • BeeMachine: The researchers launched a public web application (beemachine.ai) to allow researchers and the public to use the InceptionV3 model for real-time identification.

Discussion and Future Directions

The authors found a clear relationship between data volume and performance: species with at least 4,500 training images consistently achieved error rates below 10%. However, limitations remain. The model currently cannot identify rare or at-risk species due to a lack of training data. Future work may involve using Generative Adversarial Networks (GANs) to create synthetic training data for these rare taxa.

Additionally, some species are "cryptic," meaning they are visually indistinguishable from photos alone. Integrating geospatial data (GPS coordinates) could act as a "tie-breaker," as many similar-looking species do not overlap geographically. Finally, image quality remains a factor; accuracy drops if the bee is out of focus or diagnostic features are obscured.

Summary updated

Abstract

Pollinators are undergoing a global decline. Although vital to pollinator conservation and ecological research, species-level identification is expensive, time consuming, and requires specialized taxonomic training. However, deep learning and computer vision are providing ways to open this methodological bottleneck through automated identification from images. Focusing on bumble bees, we compare four convolutional neural network classification models to evaluate prediction speed, accuracy, and the potential of this technology for automated bee identification. We gathered over 89,000 images of bumble bees, representing 36 species in North America, to train the ResNet, Wide ResNet, InceptionV3, and MnasNet models. Among these models, InceptionV3 presented a good balance of accuracy (91.6%) and average speed (3.34 ms). Species-level error rates were generally smaller for species represented by more training images. However, error rates also depended on the level of morphological variability among individuals within a species and similarity to other species. Continued development of this technology for automatic species identification and monitoring has the potential to be transformative for the fields of ecology and conservation. To this end, we present BeeMachine, a web application that allows anyone to use our classification model to identify bumble bees in their own images.
BibTeX
@inproceedings{spiesman2021assessing,
  author = {Brian J. Spiesman and Claudio Gratton and Richard G. Hatfield and William H. Hsu and Sarina Jepsen and Brian McCornack and Krushi Patel and Guanghui Wang},
  title = {{Assessing the potential for deep learning and computer vision to identify bumble bee species from images}},
  booktitle = {Scientific},
  year = {2021},
  doi = {10.1038/s41598-021-87210-1},
}
Figure