TL;DR
This paper introduces "InvasiveAI," a novel mobile application and scalable system that leverages deep learning (CNNs and LSTMs) for automated early detection, identification, and prediction of invasive species growth. By combining multi-dimensional image analysis with geospatial forecasting, InvasiveAI significantly improves upon traditional, labor-intensive methods, offering a powerful tool for conservation efforts.
What Problem Does This Paper Address?
Invasive species pose a massive global threat, causing trillions of dollars in damages annually and contributing to biodiversity loss. Early Detection and Rapid Response (EDRR) is crucial for mitigating their spread, but current EDRR methods are often inadequate, labor-intensive, and lack predictive capabilities. Identifying and tracking invasive species typically requires expert knowledge and extensive manual effort. Existing technological solutions, like EDDMapS, primarily focus on reporting occurrences without offering automated identification or forecasting future spread. This paper addresses the critical need for a more efficient, accurate, and predictive system to combat invasive species.
Methods
The InvasiveAI system is built upon a multi-phase deep learning architecture:
- 2D Convolutional Neural Networks (CNNs): For identifying invasive species from standard 2D images (e.g., smartphone photos). The authors employed a transfer learning approach using the InceptionV3 model, pre-trained on the iNaturalist dataset. To address data scarcity for some species, they utilized image augmentation and Generative Adversarial Networks (GANs) to create synthetic training data. Separate CNN models were developed for four categories: pathogen, insect, plant, and wildlife.
- 3D Convolutional Neural Networks (CNNs): To detect invasive species in 3D point cloud data, enabling aerial and remote sensing applications. A novel implementation of the Stanford 3D PointNet architecture was developed, processing millions of cloud points from various environmental conditions and growth stages. This allows for detection in scenarios not feasible with 2D methods.
- Geospatial Long Short-Term Memory (LSTM) Models: To predict the future environmental spread of invasive species. These models integrate various parameters, including:
- Predicted New Occurrences: Estimated using SARIMAX models to forecast short-term (2-week, 3-month, 1-year) outbreaks.
- Clustering: HDBSCAN algorithm to group species occurrences and understand their spatial movement patterns.
- Weather Patterns: Incorporating humidity, temperature, rainfall, and wind speed data.
- Predation/Competition: Quantified using the Haversine distance formula to assess interactions with similar species.
These components are integrated into a mobile application and cloud-based system for real-time analysis and prediction.
Datasets
- iNaturalist Dataset: A crowd-sourced dataset of approximately 45,090 research-grade images for 31 invasive species, scraped from the iNaturalist forum. This was used for training the 2D-CNN models.
- Plant3D Dataset: A dataset of 3D point clouds compiled by the Salk Institute for Biological Studies, used for training the 3D-CNN models.
- Bugwood Database: Provided a list of invasive species, from which the top 31 high-impact species were selected. The Bugwood API was used to compile latitude and longitude occurrences for geospatial LSTM training.
- Custom Native Plant Dataset: A dataset of 63 native plant species (primarily from Texas and the US) with 28,841 images, scraped from iNaturalist, used for creating a specific 2D-CNN model for in-field testing.
Key Insights and Ideas
- Synergistic Multi-Modal Detection: Combining 2D (mobile) and 3D (aerial/remote sensing) detection methods provides a more comprehensive approach to identifying invasive species across different scenarios.
- Predictive Power of LSTMs: Integrating diverse environmental and ecological factors into LSTMs allows for robust forecasting of invasive species spread, moving beyond simple occurrence mapping.
- GANs for Data Augmentation: The successful use of GANs to generate synthetic training data for species with limited image availability is a significant contribution, addressing a common challenge in deep learning for specialized domains.
- Dynamic System for EDRR: The cloud-based system that continuously retrains models with new reportings creates a dynamic and adaptive EDRR tool, crucial for an ever-evolving threat.
- Field Validation is Key: Conducting real-world field studies with the mobile application demonstrates the practical utility and accuracy of the developed models in realistic conditions.
Key Results
- 2D-CNN Performance: The combined 4-model average achieved a Test Accuracy (TA) of 93.52% and a Test Loss (TL) of 0.2142 on the iNaturalist dataset. This is reported to be 25.78% greater than the SOTA InceptionResNetV2 accuracy on a similar dataset.
- 3D-CNN Performance: Achieved a near-perfect 97.78% validation accuracy and a 94.47% training accuracy, with a validation loss of 0.7303.
- Field Validation:
- The 2D-CNN model demonstrated an 87.5% field accuracy on invasive plants in a study at the University of Texas at Austin (BFL).
- In a study on native plants at the Lady Bird Johnson Wildflower Center (LBJWC), the 2D-CNN achieved 90% accuracy in identifying them as non-invasive.
- Mobile Application: The InvasiveAI app successfully integrates identification, reporting, and future spread visualization, providing a user-friendly interface for EDRR.
Limitations and Future Work
- Limitations:
- The 3D CNNs are not directly integrated into the mobile application due to computational constraints, but are intended for use with LIDAR and drone systems.
- The paper notes that for native species, the 2D-CNNs sometimes show low confidence, necessitating a threshold to avoid false positives.
- Future Work:
- Develop a prediction/detection API accessible by drone/rover systems.
- Increase the number of invasive species that can be detected.
- Expand the application's reach to Environmental Services worldwide.
- Further integration of 3D CNN capabilities into more accessible platforms.
Summary updated