Towards a standardized framework for AI-assisted, image-based monitoring of nocturnal insects

David B. Roy, Jamie Alison, Tom August, Marc Bélisle, Kim Bjerge, Joseph J. Bowden, Michael James Bunsen, Fagner Cunha, Quentin Geissmann, Katriona Goldmann, Alba Gomez-Segura, Anuj Jain, Chantal M. Huijbers, Maxim Larrivée, Jordann Lawson, Hjalte M. R. Mann, Marc J. Mazerolle, Kent P. McFarland, Léonard Pasi, Scott J. Peters, Nathan Pinoy, David Rolnick, Grace Skinner, O. T. Strickson, Asger Svenning, Simon Teagle, Toke T. HøyePhilosophical 2024 · DOI

TL;DR

This paper proposes a standardized framework for AI-assisted, image-based monitoring of nocturnal insects, crucial for understanding global biodiversity trends. It outlines a complete workflow from sensor design and deployment to data processing and analysis, emphasizing the need for robust machine learning models that account for inherent biases in automated data collection.

What Problem Does This Paper Address?

The world faces a biodiversity crisis, and insects, vital for ecosystem services, are experiencing alarming declines. However, monitoring insect populations is challenging due to their vast diversity, the scarcity of experts, and the limitations of traditional methods. Existing monitoring data is often biased towards a few regions and insect groups. This paper addresses the urgent need for scalable, standardized, and cost-effective methods to monitor insect biodiversity globally, particularly for nocturnal insects, which are often overlooked. The authors aim to bridge the gap between technological advancements in AI and ecological monitoring needs by proposing a comprehensive framework.

Methods

The paper details a machine learning workflow for analyzing images from automated camera traps. This workflow comprises four key steps:

  1. Insect Detection: Utilizing the Segment Anything Model (SAM) to segment insects from images, followed by manual correction of errors. A synthetic dataset is then generated by pasting segmented insects onto empty backgrounds with augmentations to train deep learning models. This approach aims to overcome challenges like overlapping insects, inconsistent lighting, and contamination.
  2. Moth/Non-Moth Classification: A binary classifier (e.g., ResNet-50) is trained to distinguish moths from other nocturnal arthropods attracted to lights. This step is crucial for filtering images for subsequent, more detailed classification.
  3. Species-Level Classification: This is the most challenging step, involving fine-grained classification of moth species. The authors discuss approaches that leverage existing large datasets and employ techniques like domain adaptation and data augmentation to handle class imbalance (where some species have many images, and others very few).
  4. Tracking Individual Moths: To estimate population sizes, the framework includes methods for tracking individual moths across consecutive frames. This involves algorithms that consider factors like intersection over union (IoU), crop size, distance between bounding boxes, and feature similarity.

The framework also emphasizes the importance of metadata and standards for defining data creation and ensuring contextual information is stored, enabling interoperability and impact.

Datasets

The paper discusses the use of existing datasets and the creation of new ones:

  • Existing Labeled Datasets: The authors highlight the need for extensive labeled training data, which is a bottleneck. They suggest leveraging existing datasets from citizen science projects and museums, such as those indexed by the Global Biodiversity Information Facility (GBIF). GBIF indexes over 31 million images containing insects.
  • Synthetic Datasets: For insect detection, a synthetic dataset is created by augmenting images of detected insects.
  • GBIF Data: A pipeline is described for obtaining data from GBIF, comparing it against the GBIF taxonomic backbone, and removing duplicates. For example, regions like Quebec-Vermont and UK-Denmark yielded over 600,000 and 990,000 images, respectively.
  • Expert-Annotated Data: While acknowledging the value of expert-annotated data, the paper also aims to develop approaches that require minimal additional labeling for new locations or hardware.

Key Insights and Ideas

  • The Power of a Standardized Framework: The paper argues that a unified framework, from sensor design to data publishing, is essential for scaling up insect monitoring and achieving global comparability.
  • AI as a Game-Changer for Nocturnal Insects: Automated, AI-assisted camera systems offer a scalable and cost-effective solution to overcome the limitations of traditional methods for monitoring nocturnal insects, which are often understudied.
  • Addressing Data Biases is Crucial: Automated systems, while powerful, introduce their own biases (e.g., imperfect detection, sampling schedules). The framework emphasizes the need for statistical models that explicitly account for these biases to ensure accurate ecological inferences.
  • Leveraging Existing Data: The paper champions the use of large, existing biodiversity datasets like GBIF to train machine learning models, reducing the burden of manual data annotation.
  • Edge AI for Efficiency: The authors suggest that processing data closer to the sensor (edge AI) can help manage power and data storage constraints, enabling real-time analysis and adaptive sampling.

Key Results

  • High Accuracy in Classification: A binary classifier for moth/non-moth differentiation achieved an accuracy of 96% on a held-out set.
  • Species Classification Performance: For species-level classification, models achieved an accuracy of 86.1% on a held-out GBIF test set and 77.8% on a small expert-labelled set of camera trap data for over 3000 species in the Quebec-Vermont region.
  • Improved Insect Detection: The proposed deep learning approach for insect detection significantly outperforms previous models, with improved ability to detect overlapping moths and requiring less manual labeling.
  • Feasibility Demonstrated: The paper describes the Automated Monitoring of Insects (AMI) system, which has been deployed and powered by solar energy, demonstrating the practical feasibility of the proposed approach.

Limitations and Future Work

  • Limitations:
    • Imperfect Detection: Acknowledged as a significant challenge, leading to potential biases in abundance and occurrence estimates.
  • Class Imbalance: The vast number of moth species and the uneven distribution of training images per species pose a challenge for accurate classification.
  • Hardware and Deployment Constraints: Practical limitations on where systems can be deployed (accessibility, security) can lead to spatio-temporal biases.
  • Sublethal Impacts: The use of light to attract moths can have sublethal effects, which need further investigation.
  • Future Work:
    • Multidisciplinary Teams: Building teams with expertise in engineering, entomology, computer vision, and statistics is crucial.
  • Standardized Protocols: Developing guidelines for camera network deployment, data collection, and metadata standards.
  • Labeled Datasets: Creating and publishing balanced training and test datasets to improve and validate algorithms.
  • Algorithm Adaptability: Developing computer vision algorithms that are adaptable to different species, geographical contexts, and hardware.
  • Ethical Considerations: Addressing data privacy and the impact of monitoring on insect populations.
  • International Collaboration: Fostering a global network for insect monitoring and knowledge sharing.
  • Statistical Modeling: Developing models that integrate information on detection and classification probabilities to estimate unbiased state variables.

Summary updated

Abstract

Automated sensors have potential to standardize and expand the monitoring of insects across the globe. As one of the most scalable and fastest developing sensor technologies, we describe a framework for automated, image-based monitoring of nocturnal insects-from sensor development and field deployment to workflows for data processing and publishing. Sensors comprise a light to attract insects, a camera for collecting images and a computer for scheduling, data storage and processing. Metadata is important to describe sampling schedules that balance the capture of relevant ecological information against power and data storage limitations. Large data volumes of images from automated systems necessitate scalable and effective data processing. We describe computer vision approaches for the detection, tracking and classification of insects, including models built from existing aggregations of labelled insect images. Data from automated camera systems necessitate approaches that account for inherent biases. We advocate models that explicitly correct for bias in species occurrence or abundance estimates resulting from the imperfect detection of species or individuals present during sampling occasions. We propose ten priorities towards a step-change in automated monitoring of nocturnal insects, a vital task in the face of rapid biodiversity loss from global threats. This article is part of the theme issue 'Towards a toolkit for global insect biodiversity monitoring'.
BibTeX
@inproceedings{roy2024towards,
  author = {David B. Roy and Jamie Alison and Tom August and Marc Bélisle and Kim Bjerge and Joseph J. Bowden and Michael James Bunsen and Fagner Cunha and Quentin Geissmann and Katriona Goldmann and Alba Gomez-Segura and Anuj Jain and Chantal M. Huijbers and Maxim Larrivée and Jordann Lawson and Hjalte M. R. Mann and Marc J. Mazerolle and Kent P. McFarland and Léonard Pasi and Scott J. Peters and Nathan Pinoy and David Rolnick and Grace Skinner and O. T. Strickson and Asger Svenning and Simon Teagle and Toke T. Høye},
  title = {{Towards a standardized framework for AI-assisted, image-based monitoring of nocturnal insects}},
  booktitle = {Philosophical},
  year = {2024},
  doi = {10.1098/rstb.2023.0108},
}
Figure