Introduction
Autonomous Underwater Vehicles (AUVs) are essential for modern oceanography, capturing vast quantities of high-resolution seafloor imagery. However, the sheer volume of data creates a bottleneck: human experts cannot label all images, and raw pixels are too computationally expensive for real-time robotic decision-making. This paper introduces a self-supervised method that leverages "free" metadata—specifically GPS coordinates and depth—to improve representation learning. By forcing a neural network to group images that are spatially or bathymetrically close, the system achieves significant boosts in classification accuracy without requiring manual labels during training.
Methodology
The core of the proposed approach is a Metadata Regularised Autoencoder. The architecture utilizes a convolutional autoencoder (based on AlexNet) to compress complex images into meaningful latent vectors. The innovation lies in a multi-part loss function:
- Reconstruction Loss ($L_{rec}$): Ensures the network can reconstruct the original image from its compressed representation.
- Horizontal Location Loss ($L_{loc}$): Regularizes the latent space based on the assumption that images taken in close proximity are likely to share visual characteristics.
- Depth Loss ($L_{dep}$): Incorporates the ecological reality that seafloor habitats are strongly influenced by depth.
To implement this, the researchers calculate "affinity matrices" for both the metadata and the latent space. By minimizing the Kullback–Leibler (KL) divergence between these matrices, the network organizes its internal representations to match the physical map of the seafloor. A contrastive mini-batch sampling strategy is also employed to ensure the network encounters a balanced mix of nearby and distant image pairs.
Dataset and Evaluation
The method was validated using the Tasmania Dataset, comprising 32,097 color seafloor images collected by the Sirius AUV. The survey covered a 1.6 × 1.7 km region with depths ranging from 28m to 96m. For evaluation purposes, 2,221 images were manually annotated into eight distinct classes, such as Kelp, High Relief Reef, and Coarse Sand.
Key Insights and Results
- Metadata as Supervision: Location and depth are recorded automatically, providing a cost-free signal to guide learning. Depth, in particular, acts as a bridge, allowing the network to regularize learning across different, non-contiguous dives.
- Performance Gains: Integrating both location and depth metadata increased image classification accuracy by an average of 10.9% across various classifiers. The largest improvement was a 15.1% boost observed with a Radial Basis Function SVM compared to a standard autoencoder.
- Robustness: The use of heavy-tailed t-distributions in the loss function makes the model robust to noise. If specific metadata does not correlate with visual features, the network learns to ignore it, preventing performance degradation.
- Unsupervised Summarization: Clustering these metadata-aware representations allowed for the automatic selection of representative images that accurately summarized the survey, successfully identifying rare habitats that standard methods often overlook.
Conclusion and Future Work
While highly effective, the method's performance is tied to the accuracy of the AUV's navigation suite. Future work may involve incorporating additional sensor data, such as acoustic backscatter or water temperature, to further refine representations. The resulting compact, meaningful vectors are ideal for low-latency, on-robot processing in future autonomous missions.
Summary updated