Deep reinforcement learning for integrated vessel path planning with safe anchorage allocation

Gil-Ho Shin, Hyun Jong YangBrodogradnja 2025 · DOI

Abstract

This study addresses vessel path planning and anchorage allocation through a reinforcement learning approach. To improve maritime safety and efficiency, we developed an integrated system that combines Deep Q-Network and Artificial Potential Field concepts for path generation. The model implements a specialized grid extension method that accounts for actual vessel dimensions and wind direction, while incorporating differentiated safety distances for each anchorage area. Experimental validation using Automatic Identification System (AIS) data demonstrated that the system successfully generated efficient routes while maintaining all safety distance requirements during both navigation and anchoring phases. Additionally, the system ensured practicality through path simplification using the Douglas-Peucker algorithm while maintaining safety standards. The visualized optimal paths enhance navigational guidance, thereby improving both maritime traffic safety and port operational efficiency.

Introduction

Managing maritime traffic in major global hubs like South Korea's Busan Port is a massive logistical and safety challenge. Vessel Traffic Services (VTS) operators must manually assign anchorage spots and monitor routes for thousands of massive ships, a process highly susceptible to cognitive overload, communication barriers, and human error.

While previous studies have applied reinforcement learning (RL) to ship routing, they often oversimplify the problem. They typically treat massive cargo ships as single points on a map, ignore how wind affects an anchored ship's orientation, and fail to account for the complex, tiered safety distance regulations that govern actual port operations. This paper bridges the gap between theoretical RL pathfinding and the strict, physical realities of maritime traffic control.

Comparison of existing literature on vessel path planning using reinforcement learning.

Methodology

The authors developed an integrated system using a Deep Q-Network (DQN) for decision-making, combined with several clever engineering adaptations for the maritime domain:

  • APF-Inspired Reward Function: The reward structure borrows from Artificial Potential Field (APF) theory. The agent is "attracted" to the target anchorage spot (positive rewards for getting closer) and "repelled" by obstacles and other ships (heavy penalties for breaching safety margins).
  • Wind-Aware Grid Extension: Instead of representing an anchored ship as a single 60x60m grid cell, the system dynamically extends the occupied grid space based on the ship's actual Length Overall (LOA) and aligns this extension with the prevailing wind direction (since anchored ships naturally align their bows with the wind).
  • Target Position Optimization: If a human-assigned anchorage spot violates safety rules, the system automatically calculates the nearest mathematically safe alternative before routing the ship.
  • Douglas-Peucker Path Smoothing: Because grid-based RL agents tend to move in jagged, zigzag patterns that are impossible for a massive ship to sail, the authors applied the Douglas-Peucker algorithm as a post-processing step. This simplifies the trajectory into smooth, navigable straight lines and diagonal turns.

Experimental Setup and Datasets

The study utilized real-world Automatic Identification System (AIS) data from Busan Port, collected over a seven-day period (June 3–9, 2023). The dataset included 115 vessels across five distinct anchorage zones (N-1 to N-5). For the core experiments, five specific vessels of varying sizes (from a 651-ton fishing vessel to a 35,832-ton general cargo ship) were selected to compare the RL agent's optimized paths against the actual historical paths taken by human captains.

Geographical scope and anchorage zones (N-1 to N-5) in Busan Port used for the study.

Key Results and Insights

The RL agent proved it wasn't just a greedy distance-minimizer. In several scenarios, the algorithm intentionally generated longer routes than the human captains to ensure strict adherence to safety buffers around buoys and neighboring vessels.

  • Zero Safety Violations: The optimized paths reduced safety violations from 6 (in the actual human-navigated historical paths) to 0.
  • Improved Efficiency: Despite taking detours for safety, the system achieved an overall path length reduction of ~9.6% (saving over 3.5 km across the five test cases), indicating significant potential for fuel and time savings.
  • Zone-Specific Adaptability: In anchorage N-3, the system found a massive 43.6% reduction in path length by fixing an inefficient human route. Conversely, in anchorages N-2 and N-4, the system increased path lengths by 4.55% and 7.65%, respectively, specifically to detour around obstacles and enforce safety boundaries.

The baseline AIS data revealed that human captains frequently (and likely unknowingly) violated safety distance rules—such as anchoring too close to boundaries or other ships. This highlights a critical need for automated VTS assistance. By extending the grid representation to account for a ship's physical length and wind-driven orientation, the simulation avoids the "point-mass" fallacy. This prevents the RL agent from plotting a route that looks safe on paper but would clip the bow of a 300-meter cargo ship in reality.

Limitations and Future Work

The authors note a few limitations to be addressed in future iterations:

  • Anchor Chain Physics: The model used the ship's bow as the center of the safety zone but did not explicitly calculate the length of the deployed anchor chain, which affects a ship's swing radius.
  • Conservative Space Allocation: The grid extension used the maximum ship length present in a zone rather than individual ship lengths, leading to slightly inefficient use of anchorage space.
  • Computational Overhead: The detailed grid modeling and DQN iterations require significant compute power, making real-time, split-second VTS implementation challenging with current hardware.
  • Weather Variations: Future work should integrate extreme seasonal weather conditions (like typhoons or fog) and explore hybrid algorithms to speed up processing for real-time dynamic decision-making.

Conclusion

This research demonstrates that integrating DQN with physical constraints like wind direction and vessel dimensions significantly improves the safety and efficiency of port operations. By automating path planning and anchorage allocation, the system provides a robust tool for VTS operators to manage complex maritime traffic while minimizing human error.

Summary updated

Abstract

This study addresses vessel path planning and anchorage allocation through a reinforcement learning approach. To improve maritime safety and efficiency, we developed an integrated system that combines Deep Q-Network and Artificial Potential Field concepts for path generation. The model implements a specialized grid extension method that accounts for actual vessel dimensions and wind direction, while incorporating differentiated safety distances for each anchorage area. Experimental validation using Automatic Identification System (AIS) data demonstrated that the system successfully generated efficient routes while maintaining all safety distance requirements during both navigation and anchoring phases. Additionally, the system ensured practicality through path simplification using the Douglas-Peucker algorithm while maintaining safety standards. The visualized optimal paths enhance navigational guidance, thereby improving both maritime traffic safety and port operational efficiency.
BibTeX
@inproceedings{shin2025deep,
  author = {Gil-Ho Shin and Hyun Jong Yang},
  title = {{Deep reinforcement learning for integrated vessel path planning with safe anchorage allocation}},
  booktitle = {Brodogradnja},
  year = {2025},
  doi = {10.21278/brod76305},
}
Figure