Abstract
Effective maritime surveillance requires the integration of heterogeneous data sources, primarily the Automatic Identification System (AIS) and Closed-Circuit Television (CCTV). However, associating digital AIS identities with visual targets in video feeds is challenging due to coordinate disparities, sampling asynchronicity, and environmental noise. This paper presents GMvA, a graph learning-driven multi-vessel association framework. GMvA models vessel interactions as dynamic graphs and utilizes spatial-temporal attention mechanisms to capture complex motion patterns. Furthermore, an uncertainty-aware fusion module is introduced to handle measurement noise and occlusions. Evaluated on real-world datasets from the Huangpu River, GMvA achieves an association accuracy of 92.28%, demonstrating superior robustness in crowded waterways and scenarios with missing data.
Introduction
Modern maritime management relies on a combination of digital tracking and visual monitoring. The Automatic Identification System (AIS) provides precise GPS-based coordinates and vessel identification, while CCTV systems offer high-fidelity visual evidence for safety and security. Fusing these two modalities—matching a digital AIS signature to a specific physical vessel on screen—is a critical task for maritime intelligence.
Despite its importance, multi-vessel association faces several hurdles. AIS and CCTV data differ in dimensionality and sampling rates. Environmental factors, such as vessel occlusions in crowded ports and unstable AIS signals, further complicate the matching process. Traditional distance-based methods often fail when vessels are in close proximity or when projection errors occur during coordinate transformation.
Methodology: The GMvA Framework
The proposed Graph learning-driven Multi-vessel Association (GMvA) method addresses these challenges through a multi-stage pipeline designed to learn robust spatiotemporal features and account for data uncertainty.
1. Coordinate Unification and Preprocessing
The first step involves mapping CCTV pixel coordinates into the real-world AIS coordinate system. A Linear Regression Model (LRM) is employed to establish a unified reference frame. Simultaneously, AIS data is cleaned and interpolated to ensure temporal alignment with the video frames.
2. Spatiotemporal Graph Learning
Rather than treating vessels as independent entities, GMvA constructs dynamic graphs where nodes represent vessel states and edges capture temporal and spatial relationships.
- Temporal Graph Attention (TGA) Layer: This layer models the evolution of individual vessel trajectories over time, capturing motion dynamics.
- Spatial-Temporal Attention (STA) Block: This module uses multi-head attention to analyze interactions between multiple vessels. By considering the "social context" of the waterway—how vessels move relative to one another—the system can better distinguish between similar trajectories in high-density areas.
3. Uncertainty-Aware Fusion (MLP-UMF)
A key innovation of GMvA is the MLP-based Uncertainty Fusion Module. Instead of producing a deterministic similarity score, the module models the similarity between AIS and video targets as a Gaussian distribution. By predicting both a mean similarity and a variance (uncertainty), the system can effectively penalize low-confidence matches caused by noise or occlusions, enhancing overall robustness.
4. Global Association
The final association is performed using the Hungarian algorithm, which optimizes the global matching matrix to ensure a consistent one-to-one correspondence between the two datasets.
Experimental Evaluation
The researchers evaluated GMvA using real-world data collected from the Huangpu River in Shanghai, China. The dataset includes various traffic densities (Low, Moderate, and High) and manually annotated ground truth for validation.
Key Results
GMvA demonstrated significant performance gains over baseline models, including LSTMs and Transformers:
- Accuracy: GMvA achieved an average accuracy of 92.28%, outperforming the Transformer baseline (82.90%).
- Density Robustness: In high-density scenarios with approximately 30 vessels, GMvA maintained an accuracy of 86.73%, whereas traditional methods like Dynamic Time Warping (DTW) dropped to 27.64%.
- Resilience to Data Loss: The graph-based approach allows the model to maintain 45.45% accuracy even when 10 consecutive AIS records are missing, a scenario where most other models fail completely.
Conclusion
GMvA provides a robust solution for maritime multi-vessel association by leveraging graph learning and uncertainty modeling. By treating the waterway as a dynamic system of interacting entities rather than isolated points, the framework achieves high accuracy even in challenging, real-world conditions. Future work will focus on integrating radar data for further redundancy and developing online learning capabilities for real-time adaptation to new environments.
Summary updated