A Real-Time Vessel Detection and Tracking System Based on LiDAR

Liangjian Qi, Lei Huang, Yi Zhang, Yue Chen, Jianhua Wang, Xiaoqian ZhangSensors 2023 · DOI

TL;DR

This paper presents a real-time system for detecting and tracking vessels on waterways using LiDAR point cloud data. By integrating statistical filtering with Euclidean clustering for detection and employing an L-shape fitting approach for tracking, the system significantly improves accuracy and robustness compared to traditional image-based methods, especially in challenging conditions like ripples and occlusion.

What Problem Does This Paper Address?

Monitoring vessel traffic on waterways is crucial for safety and efficiency. However, conventional vision-based systems struggle with issues like poor lighting, adverse weather, target occlusion, and ID switching, leading to unreliable detection and tracking. These limitations are particularly problematic in busy river traffic scenarios. The authors aim to overcome these challenges by leveraging the rich 3D spatial information provided by LiDAR, which is less susceptible to lighting and weather conditions.

Methods

The proposed system consists of two main stages: vessel detection and vessel tracking.

Vessel Detection: * Ripples Mitigation: To address the interference from water ripples, which can distort vessel dimensions, the system first identifies and removes the horizontal plane of the ripples. This is achieved by analyzing the point cloud to define the ripple plane and then removing points within a certain height range at the bottom of the vessel. * Clustering: Euclidean clustering is employed to segment individual vessels from the point cloud. This method was chosen for its efficiency in processing high-dimensional data and its speed, which is critical for real-time applications. Statistical filtering is integrated with Euclidean clustering to further enhance noise removal and improve detection accuracy. * L-shape Fitting: After clustering, an L-shape fitting technique is used to obtain a more accurate 3D bounding box for each detected vessel. This method helps in precisely determining the vessel's size and orientation, providing a robust foundation for the subsequent tracking stage.

Vessel Tracking: * Efficient Tracking Algorithm: A simple yet efficient tracking algorithm is proposed. It utilizes the information from the detected vessels (size, position, orientation) and employs a matching degree calculation based on changes in size and position between frames. * ID Switching Reduction: The algorithm is designed to minimize target ID switching, a common problem in multi-object tracking, especially during occlusions. By carefully associating detected objects with existing tracks, it maintains consistent identities for vessels.

Datasets

The study utilized a dataset acquired from the Lingqu Canal using RS-LiDAR-M1 sensors. The data consisted of PCAP files, which were converted to PCD (point cloud data) files. Approximately 50 PCAP files, containing about 24,000 frames, were used for the experiment. The paper does not mention the introduction of a new benchmark dataset.

Key Insights and Ideas

  • LiDAR's Advantage in Waterways: The paper strongly advocates for LiDAR's superiority over camera-based systems for vessel monitoring due to its inherent 3D sensing capabilities and resilience to environmental factors.
  • Ripples as a Significant Challenge: The authors highlight water ripples as a non-trivial issue that can significantly impact vessel dimension estimation, and their proposed filtering method is a key contribution to addressing this.
  • L-shape Fitting for Robustness: The L-shape fitting approach for bounding box generation is shown to be more effective than simple rectangular fitting, especially for vessels not perfectly aligned with the sensor's axes.
  • Simplicity in Tracking: The paper demonstrates that a well-designed, simpler tracking algorithm can achieve competitive results, avoiding the complexity and computational overhead of more advanced deep learning-based trackers, which is crucial for real-time deployment.

Key Results

  • Detection Accuracy: The proposed detection method achieved an improvement of 3.3% to 8.3% compared to three conventional algorithms.
  • Tracking Performance: The system demonstrated improved multiple object tracking accuracy (MOTA) and a reduction in ID switch times and missed detections compared to three traditional tracking algorithms. Specifically, the MOTA was 1.4% to 10.1% higher than other methods.
  • ID Switches: The proposed algorithm achieved only one target ID switch, significantly outperforming other methods.
  • False Positives/Negatives: The algorithm showed a favorable balance of false positives and negatives, with the lowest number of false negatives among the compared algorithms.
  • Real-time Performance: Euclidean clustering, a core component of the detection pipeline, achieved a processing speed of 16.2 ms per frame, contributing to the system's real-time capabilities.

Limitations and Future Work

  • Limited Range: A key limitation is the range of the LiDAR sensor. As the distance increases, the point cloud density decreases, making it difficult to detect distant vessels. The authors suggest using longer-range LiDAR systems like FMCW LiDAR for future work.
  • Ripples Under Strong Conditions: While the ripple removal is effective, it can be less precise under very strong wind and wave conditions. Further optimization of the clustering algorithm or exploring deep learning for ripple removal is suggested.
  • Visualization: The current visualization capabilities are basic. The authors propose using QT or third-party LiDAR vendor software for enhanced visualization of the full river channel status.
  • Deep Learning Integration: While the paper focuses on optimized classical algorithms for real-time performance, the potential of deep learning for tasks like ripple detection and removal is acknowledged as a future research direction.

Summary updated

Abstract

Vessel detection and tracking is of utmost importance to river traffic. Efficient detection and tracking technology offer an effective solution to address challenges related to river traffic safety and congestion. Traditional image-based object detection and tracking algorithms encounter issues such as target ID switching, difficulties in feature extraction, reduced robustness due to occlusion, target overlap, and changes in brightness and contrast. To detect and track vessels more accurately, a vessel detection and tracking algorithm based on the LiDAR point cloud was proposed. For vessel detection, statistical filtering algorithms were integrated into the Euclidean clustering algorithm to mitigate the effect of ripples on vessel detection. Our detection accuracy of vessels improved by 3.3% to 8.3% compared to three conventional algorithms. For vessel tracking, L-shape fitting of detected vessels can improve the efficiency of tracking, and a simple and efficient tracking algorithm is presented. By comparing three traditional tracking algorithms, an improvement in multiple object tracking accuracy (MOTA) and a reduction in ID switch times and number of missed detections were achieved. The results demonstrate that LiDAR point cloud-based vessel detection can significantly enhance the accuracy of vessel detection and tracking.
BibTeX
@inproceedings{qi2023real,
  author = {Liangjian Qi and Lei Huang and Yi Zhang and Yue Chen and Jianhua Wang and Xiaoqian Zhang},
  title = {{A Real-Time Vessel Detection and Tracking System Based on LiDAR}},
  booktitle = {Sensors},
  year = {2023},
  doi = {10.3390/s23229027},
}
Figure