Abstract
This paper describes Georeference Contrastive Learning of visual Representation (GeoCLR) for efficient training of deep-learning Convolutional Neural Networks (CNNs). The method leverages georeference information by generating a similar image pair using images taken of nearby locations, and contrasting these with an image pair that is far apart. The underlying assumption is that images gathered within a close distance are more likely to have similar visual appearance, which is reasonably satisfied in seafloor robotic imaging applications where image footprints are limited to edge lengths of a few metres and are taken so that they overlap along a vehicle's trajectory. A key advantage of this method is that it is self-supervised and does not require any human input for CNN training. We demonstrate how the latent representations generated by GeoCLR can be used to efficiently guide human annotation efforts, where the semi-supervised framework improves classification accuracy by an average of 10.2% compared to the state-of-the-art SimCLR using the same CNN and equivalent number of human annotations for training.
Introduction
Robotic underwater surveys generate tens of thousands of images per dive, creating a massive data interpretation bottleneck. Traditional supervised learning requires extensive human annotation, which is costly and slow. Furthermore, models pre-trained on terrestrial datasets (like ImageNet) often fail in underwater environments due to unique optical properties like light attenuation and turbidity.
While self-supervised methods like SimCLR offer a path forward by learning from unlabeled data, they typically rely on artificial data augmentations (e.g., cropping, flipping) of a single image. GeoCLR introduces a novel twist: leveraging the inherent spatial structure of seafloor surveys—geography—as a "free" source of similarity to train more robust and efficient models.
Methodology: Leveraging Spatial Proximity
The core innovation of GeoCLR is the redefinition of "positive pairs" in contrastive learning. In standard frameworks like SimCLR, a positive pair consists of two different augmentations of the same image. GeoCLR, however, identifies positive pairs as images taken at physically nearby locations.
Spatial Positive Pairs and 3D Distance
The method assumes that seafloor habitats are "patchy" and span areas larger than a single image footprint (typically a few meters). To find these neighbors, the authors employ a 3D distance metric that accounts for horizontal position and depth. They include a scaling factor ($\lambda$) for depth, allowing the model to be sensitive to the fact that vertical changes often change image appearance more drastically than horizontal ones.
Machine-Prioritized Annotation ($H$-kmeans)
To further optimize human effort, the authors use hierarchical k-means ($H$-kmeans) clustering in the learned latent space. Instead of labeling images at random, experts label representative images from these clusters. This ensures that human experts spend their time labeling the most informative examples rather than redundant ones, representing the diversity of the data rather than just the diversity of the labels.
Experimental Setup
The framework was evaluated using a dataset collected by the Sirius Autonomous Underwater Vehicle (AUV) off the coast of Tasmania:
- 86,772 georeferenced images across 12 dives at depths ranging from 28 to 96 meters.
- 5,369 expert human labels across six classes (Kelp, High Relief Reef, Low Relief Reef, Reef & Sand, Screw Shell Rubble, and Sand).
- Training was performed on a single consumer-grade GPU (NVIDIA TITAN RTX), completing in approximately 26 hours, making it feasible for use between dives during field trials.
Key Results and Insights
GeoCLR demonstrates significant improvements in both efficiency and accuracy:
- Efficiency Gain: GeoCLR with just 100 human labels (0.1% of the dataset) achieved the same accuracy as state-of-the-art transfer learning using 1,000 labels, representing a 10x reduction in manual effort.
- Accuracy Boost: The method improved classification accuracy by an average of 10.2% compared to SimCLR using an equivalent number of annotations.
- Natural Augmentation: By using neighboring images, GeoCLR captures natural variability—slight changes in lighting, perspective, and water clarity—that artificial digital filters cannot easily simulate.
Conclusion
GeoCLR provides a computationally efficient, self-supervised solution for seafloor mapping. By utilizing "free" georeference data, it bridges the gap between massive raw datasets and actionable ecological insights. While hyperparameters like distance thresholds may require tuning for different environments, the framework significantly reduces the human bottleneck in marine robotics.
Summary updated