TL;DR
This paper introduces a novel system for real-time vessel speed measurement using monocular cameras, combining an enhanced YOLOv11 object detection model (M-YOLOv11) with visual tracking. The method overcomes limitations of AIS data by accurately converting pixel-based movement to physical speed, achieving an average error of less than 0.45 knots.
What Problem Does This Paper Address?
Accurate vessel speed monitoring is crucial for maritime safety and traffic management. While Automatic Identification Systems (AIS) are commonly used, they are prone to failures, delays, and signal congestion, leading to unreliable speed data. Existing vision-based methods often struggle with environmental challenges (fog, low light), complex vessel shapes, and require precise camera calibration, limiting their practical applicability. This paper aims to develop a robust, cost-effective, and widely applicable real-time vessel speed measurement system that addresses these limitations.
Methods
The proposed system integrates several key components:
- M-YOLOv11 Object Detection Model: This is an enhanced version of YOLOv11, featuring two main improvements:
- Multi-scale Depthwise Separable Convolution (MDSC) module: Designed to capture vessel features at various scales and shapes, improving detection in low-quality images.
- IP-IoU (Interior Perception Intersection over Union) loss function: This function focuses on the internal structure of the target bounding box, enhancing detection frame stability and reducing localization errors caused by shape variations or partial occlusions.
- Visual Tracking (DeepSORT): The DeepSORT algorithm is employed to track detected vessels across frames, providing consistent trajectory information. It combines a Kalman filter for state prediction with a Hungarian algorithm for data association, ensuring robust tracking even with potential ID switches.
- Coordinate Transformation: A crucial step involves establishing a mapping between image pixel coordinates and real-world coordinates. This is achieved by carefully measuring camera parameters (height, focal length, etc.) and using geometric relationships to convert pixel displacement into physical distance.
- Frame Difference Method (FDM) for Speed Calculation: Once pixel displacement is converted to physical displacement, the FDM is used. By measuring the change in a vessel's position between two frames and knowing the time interval between them, the vessel's speed is calculated. This is then converted from meters per second to knots.
- Camera Calibration: To ensure accurate mapping between pixel and world coordinates, the paper utilizes the Checkerboard Calibration Method (CCM) to correct for lens distortions (barrel and pincushion).
Datasets
- Custom Visible Vessel Dataset: The authors constructed a dataset of 3700 vessel images collected from surveillance cameras in the Jiujiang River Basin. This dataset includes images captured during daytime, nighttime, and foggy conditions to ensure the model's adaptability to diverse environments. The dataset was split into training (70%), validation (20%), and testing (10%) sets.
Key Insights and Ideas
- Beyond AIS: The research highlights the critical need for vision-based alternatives to AIS for vessel speed monitoring, especially in scenarios where AIS data is unreliable.
- M-YOLOv11's Dual Enhancement: The combination of MDSC for multi-scale feature extraction and IP-IoU for stable bounding box regression proves effective in improving detection accuracy and robustness, particularly in challenging visual conditions.
- Pixel to Physical Conversion is Key: The paper emphasizes that accurate speed measurement hinges on a precise and robust method for converting pixel-based movement to real-world distances, requiring careful camera calibration and coordinate transformation.
- Frame Difference is Sufficient with Good Tracking: While simple, the frame difference method, when combined with accurate target tracking and precise coordinate conversion, can yield highly accurate speed measurements.
Key Results
- M-YOLOv11 Performance: The M-YOLOv11 model achieved a significant improvement in average precision (AP50:95) of 13.95% over the baseline YOLOv11. It also outperformed other state-of-the-art models like EfficientDet, RetinaNet, YOLOv3, YOLOv5, YOLOv8, YOLOv10, and YOLOv11 in various evaluation metrics.
- Speed Measurement Accuracy: In practical tests with 100 vessels, over 60% of the measured speeds had errors less than 0.5 knots, and more than 90% had errors less than 1 knot. The overall average error was below 0.45 knots.
- Robustness: The system demonstrated good performance across different times of day and weather conditions, including foggy environments, showcasing its environmental adaptability.
Limitations and Future Work
- Visibility Dependence: While the M-YOLOv11 model shows improved robustness to fog, extremely dense fog conditions (visibility below 50m) still lead to increased speed measurement errors due to significant declines in camera imaging quality.
- Future Work: The paper suggests that further research could focus on improving the system's performance in extremely low visibility conditions and potentially integrating it with other sensor data for even greater accuracy and reliability.
Summary updated