A Contrastive-Learning-Based Method for the Few-Shot Identification of Ship-Radiated Noises

Leixin Nie, Chao Li, Haibin Wang, Jun Wang, Yonglin Zhang, Fan Yin, Franck Marzani, Alexis Bozorg GrayeliJournal of Marine Science and Engineering 2023 · DOI

TL;DR

This paper introduces a novel contrastive learning approach for identifying individual ships based on their radiated noise, even with very limited training data (few-shot learning). By focusing on learning discriminative features and employing a parameter-free classifier, the method significantly outperforms existing techniques in challenging ship identification scenarios.

What Problem Does This Paper Address?

Classifying ships based on their underwater radiated noise is crucial for applications like maritime security and monitoring. However, traditional methods often struggle with "ship classification" (grouping ships by type, e.g., cargo, tanker) which suffers from high intra-class variance (ships of the same type can sound very different). This paper tackles the more granular "ship identification" problem – distinguishing individual ships. This is significantly harder due to the need for finer distinctions and the inherent data scarcity problem in real-world acoustic recordings, where obtaining sufficient data for each individual ship is difficult. This "few-shot" scenario, where only a handful of examples are available per ship, poses a major challenge for data-hungry deep learning models.

Methods

The core of the proposed method is a contrastive learning framework adapted for few-shot ship identification. Instead of traditional classification where a model learns to map inputs to class labels, this approach focuses on learning a robust feature representation.

Here's a breakdown:

  • Feature Extractor (CNN): A ResNet-18 architecture is used as a Convolutional Neural Network (CNN) to extract features from time-frequency representations (spectrograms) of the ship noises. Crucially, the CNN's parameters are shared across all samples.
  • Contrastive Learning Objective: During training, the model is presented with pairs of audio samples. The objective is to:
  • Maximize similarity between features extracted from samples belonging to the same ship (positive pairs).
  • Minimize similarity between features extracted from samples belonging to different ships (negative pairs). This is achieved by optimizing a loss function (InfoNCE loss) that encourages features of similar ships to be close in an embedding space and features of dissimilar ships to be far apart.

  • Parameter-Free Classifier: In the inference (testing) phase, the trained CNN acts solely as a feature extractor. A "registration template" is created for each known ship using its training samples. To identify a new, unknown ship noise, its features are extracted and compared to all registration templates. The new noise is assigned to the ship whose template has the highest similarity score. This eliminates the need to train a separate classifier, which is beneficial in low-data regimes.

Datasets

The paper utilizes both publicly available and newly collected datasets:

  • ShipsEar Dataset: A subset of this dataset was used to create the SID1 dataset, consisting of 5 different passenger boats. This dataset was specifically designed to have significant temporal separation between training and testing samples to ensure generalization.
  • VOS (Vessel Observation by Single-hydrophone) Data: This dataset was collected from sea trials in the South China Sea. From this, two new datasets were established:
  • SID2: Comprising 3 different fishing boats (categorized by size: large, medium, small).
  • SID3: Comprising 27 different vessels, representing a more challenging, larger-scale identification task.

For all datasets, the N-way K-shot setting was employed, with K typically set to 5 (meaning 5 samples per ship for training).

Key Insights and Ideas

  • Ship Identification is Harder than Classification: The paper clearly articulates why distinguishing individual ships is a more challenging problem than classifying them into broad categories, primarily due to the need for finer feature discrimination and the exacerbation of data scarcity.
  • Contrastive Learning for Feature Robustness: The core idea of contrastive learning is highly effective here. By forcing the model to learn what makes ships different rather than just what makes them a type, it builds more robust feature representations that are less susceptible to variations in soundscapes or recording conditions.
  • Parameter-Free Classification is Key for Few-Shot: The absence of a parameterized classifier in the inference stage is a critical design choice. It bypasses the need for extensive training data for the classifier itself, which is precisely the bottleneck in few-shot scenarios.
  • Feature Space Visualization Matters: The t-SNE visualizations clearly demonstrate how the proposed method creates more separable clusters in the feature space compared to baseline methods, providing strong qualitative evidence for its effectiveness.

Key Results

  • Significant Accuracy Gains: On the 5-ship identification task (SID1), the proposed method achieved an accuracy of 0.68, significantly outperforming conventional SCNet (0.26) and a Siamese network baseline (0.49).
  • Performance Across Datasets: The method consistently performed best across different datasets and task complexities:
  • SID2 (3-ship identification): Accuracy of 0.785 (vs. SCNet 0.343, SiamNet 0.680).
  • SID3 (27-ship identification): Accuracy of 0.534 (vs. SCNet 0.151, SiamNet 0.330).
  • Impact of Training Data: Increasing the number of training samples (K) improved performance for all methods, but the contrastive methods (SiamNet and Proposed) showed less dramatic gains than SCNet, likely due to their inherent robustness and lack of a parameterized classifier.
  • Vessel-wise Performance Variation: While overall performance is strong, the paper notes that identification accuracy can vary between individual vessels, suggesting potential for further refinement.

Limitations and Future Work

  • Multi-Vessel Interference: The current work simplifies the problem by using interference-free moments. Real-world scenarios often involve overlapping noises from multiple vessels, which needs to be addressed.
  • Far-Field Noise Distortion: The study primarily uses near-field vessel data. The impact of acoustical distortion from far-field sources on identification performance requires further investigation.
  • Large-Scale Environmental Changes: While the method shows generalization across time and space variations within the same ocean environment, its performance under significant environmental changes (e.g., different ocean types) needs further study.
  • Vessel-wise Performance Inconsistency: The paper acknowledges that performance varies across individual vessels, suggesting that further research could explore ways to improve consistency.

Summary updated

Abstract

For identifying each vessel from ship-radiated noises with only a very limited number of data samples available, an approach based on the contrastive learning was proposed. The input was sample pairs in the training, and the parameters of the models were optimized by maximizing the similarity of sample pairs from the same vessel and minimizing that from different vessels. In practical inference, the method calculated the distance between the features of testing samples and those of registration templates and assigned the testing sample into the closest templates for it to achieve the parameter-free classification. Experimental results on different sea-trial data demonstrated the advantages of the proposed method. On the five-ship identification task based on the open-source data, the proposed method achieved an accuracy of 0.68 when only five samples per vessel were available, that was significantly higher than conventional solutions with accuracies of 0.26 and 0.48. Furthermore, the convergence of the method and the behavior of its performance with increasing data samples available for the training were discussed empirically.
BibTeX
@article{nie2023contrastive,
  author = {Leixin Nie and Chao Li and Haibin Wang and Jun Wang and Yonglin Zhang and Fan Yin and Franck Marzani and Alexis Bozorg Grayeli},
  title = {{A Contrastive-Learning-Based Method for the Few-Shot Identification of Ship-Radiated Noises}},
  journal = {Journal of Marine Science and Engineering},
  year = {2023},
  doi = {10.3390/jmse11040782},
}
Figure