Abstract
Training deep learning models for environmental monitoring is often bottlenecked by the massive effort required for expert manual labeling. This paper introduces a semi-supervised pipeline that uses georeference data and smart clustering to achieve 85% of the accuracy of a fully-labeled dataset using just 40 prioritized annotations—a nearly 250x reduction in human effort.
Introduction
In fields like marine biology or aerial forestry monitoring, robotic platforms such as drones and Autonomous Underwater Vehicles (AUVs) collect tens of thousands of images per mission. However, converting these images into actionable data typically requires expert scientists to manually label them—a process that is both time-consuming and expensive.
Standard AI approaches, such as fine-tuning models pre-trained on ImageNet, often underperform in these domains because seafloor or satellite imagery differs significantly from generic datasets. Furthermore, environmental datasets are frequently "unbalanced," with common classes (like sand) vastly outnumbering rare but critical features (like endangered coral). Random sampling for labeling often misses these rare classes entirely. This research addresses the challenge of training high-performing classifiers on a per-mission basis without requiring thousands of manual annotations.
The LGA-SS Pipeline
The proposed LGA-SS (Location Guided Autoencoder - Semi-Supervised) pipeline streamlines the transition from raw data collection to high-accuracy inference through four integrated stages.
1. Unsupervised Feature Learning (LGA)
The process begins with a Location Guided Autoencoder. Unlike standard autoencoders, this model is regularized by georeference data (GPS coordinates). It operates on the principle that images captured in close spatial proximity are likely to share similar features. This spatial constraint helps the model learn a "latent representation" that is highly specific to the target environment.
2. Prioritized Labelling
To maximize the impact of human effort, the system employs hierarchical $k$-means clustering within the latent space. Instead of labeling random images, the system identifies "centroids"—the most representative images of each cluster. This ensures that even rare classes are captured and presented to the expert for annotation.
3. Pseudo-labelling
After an expert labels a minimal subset (e.g., 40 images), a Radial Basis Function Support Vector Machine (R-SVM) propagates these labels to the remaining unlabelled images. This "pseudo-labelling" step creates a larger, albeit noisier, training set from a tiny human-labeled seed.
4. CNN Training and Inference
Finally, a standard Convolutional Neural Network (such as ResNet18) is trained using a combination of high-quality human labels and the generated pseudo-labels. This hybrid dataset allows the model to achieve high performance with minimal manual intervention.
Experimental Validation
The method was validated across four distinct environmental datasets with highly skewed class distributions:
- Seafloor: Color imagery from an AUV featuring classes like "Bacterial Mat," "Shell Fragment," and "Sand."
- Terrestrial Aerial (Mountain, Island, Urban): Cropped images from ESRI World Imagery representing diverse ecosystems and human infrastructure.
Key Insights
- Geography as a Self-Supervised Signal: Using spatial proximity as a regularizer allows the model to learn relevant environmental features more effectively than pixel-only analysis.
- Diversity Over Quantity: The research demonstrates that selecting which images to label is more critical than the total volume of labels. Hierarchical clustering ensures that rare but important classes are not overlooked.
- Mission-Specific Training: Because the pipeline is efficient enough to run on a single mission's data, it avoids the "transferability gap" where models trained in one geographic region fail in another.
Results
The LGA-SS method achieved accuracy comparable to standard transfer learning while using an order of magnitude fewer annotations. Notably, in the seafloor dataset, the model reached 85% of the accuracy of a fully-labeled model (10,000 labels) using only 40 prioritized annotations. Furthermore, the method successfully recovered rare classes like "Artificial Object" and "Bacterial Mat" with just 20 total labels, whereas random sampling failed to identify them at such low sample sizes.
Limitations and Future Work
While the pipeline significantly reduces human effort, it introduces computational overhead, as an autoencoder and CNN must be trained for every new mission. Additionally, the benefits of "Probabilistic Pseudo-labelling" to mitigate label noise were only evident at higher label counts ($M \approx 1000$). Future research will focus on integrating this workflow into real-time robotic platforms, allowing autonomous systems to request specific labels from operators during active missions.
Summary updated