Multi-Task Learning-Enabled Automatic Vessel Draft Reading for Intelligent Maritime Surveillance

Jingxiang Qu, Ryan Wen Liu, Chenjie Zhao, Yu Guo, Sendren Sheng-Dong Xu, Fenghua Zhu, Yisheng Lv

TL;DR

This paper introduces MTL-VDR, a multi-task learning framework designed to automate vessel draft reading—a critical task for preventing ship overloading and ensuring maritime safety. By combining real-time object detection with a specialized segmentation approach and a rule-based correction module, the system achieves high-precision draft estimation at speeds exceeding 40 FPS, even in challenging conditions like stained hulls or choppy water.

What Problem Does This Paper Address?

Vessel draft reading (VDR) measures how deep a ship sits in the water, which is the primary way to determine its load. Traditionally, this is done manually by observers or via basic computer vision that struggles with "real-world" noise: hull stains, varying light, and the irregular, moving line where the water meets the ship (the waterline).

Existing automated solutions often treat detection and segmentation as separate, heavy processes, making them too slow for real-time surveillance. Furthermore, if a draft scale (the numbers painted on the hull) is obscured by mud or rust, standard OCR often fails, leading to dangerous miscalculations.

Methods

The authors propose a pipeline called MTL-VDR, which breaks the problem into four intelligent stages:

  1. Draft Mark Detection: Uses a customized YOLOv8 architecture to locate the general area of the draft marks on the ship's hull.
  2. Multi-Task Learning (MTL) Subnet: This is the core innovation. A single backbone (DarkNet-53) feeds into two parallel "heads":
  3. Scale Recognition: An anchor-free detection head (based on YOLOX-s) that identifies individual digits and units on the hull.
  4. Vessel/Water Segmentation: A U-Net-style structure that classifies pixels as "water" or "vessel."
  5. Dynamic Loss Weighting: To train these two very different tasks (detecting boxes vs. classifying pixels) simultaneously, the authors use a loss function that weights tasks based on homoscedastic uncertainty. This prevents one task from dominating the learning process.
  6. Spatial Distribution Correction: Since draft scales follow strict physical rules (e.g., they are exactly 0.2 meters apart), the authors added a rule-based correction layer. If the AI misreads a "3.6" as an "8," the system uses the surrounding visible scales to "math out" the correct value.
  7. Adaptive Depth Estimation: A final geometric calculation that fits a line to the waterline and compares it to the recognized scales to produce a final depth in meters.

Datasets

The authors constructed and released a new benchmark called the VDR dataset:

  • Detection: 2,226 images featuring various vessel types (cargo, passenger, patrol).
  • Recognition/Segmentation: 1,198 finely annotated image patches.
  • Evaluation: 45 high-stakes test cases, specifically split into 30 "clear" images and 15 "stained" images to test robustness against real-world grime.

Key Insights and Ideas

  • The Waterline is the Weak Link: Standard segmentation losses (like Cross-Entropy) treat every pixel equally. The authors realized that for draft reading, the only pixels that truly matter are the ones at the very top of the water boundary. They introduced Mean Absolute Vertical Distance (MAVD) loss to force the model to be hyper-accurate at that specific interface.
  • Physics as a Prior: Pure deep learning can make "silly" mistakes (like skipping a number in a sequence). By encoding the maritime regulation that scales are 0.1m high and 0.2m apart into a post-processing algorithm, they created a "fail-safe" for the neural network.
  • Efficiency through Sharing: By using a shared backbone for both digit recognition and water segmentation, the model reduces redundant computations, allowing it to run at 60+ FPS on a mid-range laptop GPU (RTX 3060).

Key Results

  • Accuracy: The system achieved a Mean Absolute Draft Depth Error (MADDE) of 0.074 meters. For context, competing methods like Faster R-CNN or standard U-Net combinations had errors ranging from 0.46m to 1.1m.
  • Speed: The inference time is approximately 0.016 seconds per image, comfortably exceeding the 30 FPS requirement for real-time maritime monitoring.
  • Robustness: In tests on "stained" hulls where digits were partially obscured, the spatial correction method allowed the system to maintain high accuracy while other OCR-based methods failed completely.

Limitations and Future Work

  • Environmental Factors: The current model is trained on clear daylight data. The authors acknowledge that low-visibility conditions (fog, heavy rain, or night) will require additional "image enhancement" modules to be reliable.
  • Unit Systems: The current logic is hard-coded for the metric system used in inland Chinese waterways. Global deployment would require adapting the spatial rules for imperial units (feet/inches) used in other regions.
  • Oceanic vs. Inland: The paper focuses on inland vessels. Oceanic conditions involve much larger waves and different hull curvatures, which may require more complex geometric fitting than the linear model proposed here.

Summary updated

Abstract

The accurate and efficient vessel draft reading (VDR) is an important component of intelligent maritime surveillance, which could be exploited to assist in judging whether the vessel is normally loaded or overloaded. The computer vision technique with an excellent price-to-performance ratio has become a popular medium to estimate vessel draft depth. However, the traditional estimation methods easily suffer from several limitations, such as sensitivity to low-quality images, high computational cost, etc. In this work, we propose a multi-task learning-enabled computational method (termed MTL-VDR) for generating highly reliable VDR. In particular, our MTL-VDR mainly consists of four components, i.e., draft mark detection, draft scale recognition, vessel/water segmentation, and final draft depth estimation. We first construct a benchmark dataset related to draft mark detection and employ a powerful and efficient convolutional neural network to accurately perform the detection task. The multi-task learning method is then proposed for simultaneous draft scale recognition and vessel/water segmentation. To obtain more robust VDR under complex conditions (e.g., damaged and stained scales, etc.), the accurate draft scales are generated by an automatic correction method, which is presented based on the spatial distribution rules of draft scales. Finally, an adaptive computational method is exploited to yield an accurate and robust draft depth. Extensive experiments have been implemented on the realistic dataset to compare our MTL-VDR with state-of-the-art methods. The results have demonstrated its superior performance in terms of accuracy, robustness, and efficiency. The computational speed exceeds 40 FPS, which satisfies the requirements of real-time maritime surveillance to guarantee vessel traffic safety.
BibTeX
@article{qu2024multi,
  author = {Jingxiang Qu and Ryan Wen Liu and Chenjie Zhao and Yu Guo and Sendren Sheng-Dong Xu and Fenghua Zhu and Yisheng Lv},
  title = {{Multi-Task Learning-Enabled Automatic Vessel Draft Reading for Intelligent Maritime Surveillance}},
  journal = {IEEE Trans. Intell. Transp. Syst.},
  year = {2024},
  url = {https://openreview.net/forum?id=fLTwdjsfWL},
}

Loading related content...

Figure