Navigating maritime vessels through complex environments requires balancing goal-oriented movement with robust obstacle avoidance. While Deep Reinforcement Learning (DRL) has shown success in robotics, its application to autonomous ships often suffers from poor generalization—an agent trained in one harbor may fail in another due to overfitting to specific geographical geometries. This paper introduces the Continuous Vessel Navigator (CVN), a framework designed to bridge the gap between simulation and real-world versatility through continuous control and a novel data augmentation strategy.
The Generalization Challenge
Traditional maritime navigation models often rely on discrete action spaces (e.g., fixed turning increments), which lack the precision required for smooth vessel control. More importantly, these models struggle to generalize to unseen maps. To address this, the researchers propose a system that combines the Deep Deterministic Policy Gradient (DDPG) algorithm with a focus on local environmental features rather than global map memorization.
Methodology: The CVN Framework
Local View Maker
Instead of processing a massive global map, the agent utilizes a Local View Maker. This component generates a 51x51 pixel local representation centered on the vessel. By focusing on immediate surroundings—identifying land (brown pixels) and the target (green pixels)—the agent learns universal navigation principles rather than specific routes.
State-Action Rotation (SAR)
The core innovation of this work is State-Action Rotation (SAR). This data augmentation technique leverages the inherent symmetry of navigation. When the agent experiences a transition (State $\rightarrow$ Action $\rightarrow$ New State), the system rotates that experience by 90°, 180°, and 270°, storing all four versions in the replay buffer. This teaches the agent that the logic of avoiding an obstacle remains constant regardless of the ship's orientation.
Path Simplification with LARDP
To guide the DRL agent, the system uses a modified path-simplification algorithm called Land Attended Rammer-Douglas-Peucker (LARDP). Standard algorithms often place waypoints on land when simplifying a complex path. LARDP ensures that intermediate goals remain in navigable water, preventing the agent from being led into "trap" areas near shorelines.
Key Results and Insights
The researchers evaluated the CVN model by training it on geographical data from Halifax and testing it on the unseen environment of Montreal. The results demonstrated significant performance gains:
- Generalization Success: The CVN model with SAR achieved a 30.82% improvement in the Rate of Arrival To Destination (RATD) on the unseen Montreal map compared to baseline models.
- Consistency: SAR consistently improved success rates by nearly 12% across various scenarios, proving that symmetry-based augmentation effectively reduces overfitting.
- Training Dynamics: Interestingly, the study found that training with larger distances between start and end points improved performance. Longer trips forced the agent to navigate complex straits more frequently, providing richer training data.
Conclusion
By combining continuous control with symmetry-aware data augmentation, the CVN framework significantly enhances the ability of autonomous vessels to navigate unfamiliar maritime environments. Future work will focus on integrating dynamic collision avoidance (COLREGs) and transitioning from map-based inputs to real-time sensor fusion involving LiDAR and camera feeds.
Summary updated