The Contributions of AI Usage Over AUVs’ Path-Following and Path-Planning

Abeer Ali Sirelkhatim, Muhammad Ali, Bei PengInternational Journal of Computer Applications 2025 · DOI

TL;DR

This paper reviews the transformative role of Reinforcement Learning (RL) and Deep Reinforcement Learning (DRL) in helping Autonomous Underwater Vehicles (AUVs) master path-following and path-planning. By moving beyond rigid traditional control theories, AI allows these "underwater drones" to adapt to the chaotic, nonlinear, and unpredictable environments of the deep ocean.

What Problem Does This Paper Address?

The ocean is a nightmare for control engineers. AUVs face a "triple threat" of technical challenges:

  1. High Nonlinearity: Their movement is governed by complex hydrodynamic forces that change with speed and depth.
  2. Underactuation: Most AUVs have fewer motors (actuators) than they have degrees of freedom (DoF), making them inherently difficult to steer.
  3. Environmental Uncertainty: Unlike a factory floor, the ocean is full of unpredictable currents, shifting obstacles, and "poorly known" models.

Traditional controllers (like PID or Lyapunov-based systems) often struggle when the math gets messy or the environment changes. This paper explores how AI—specifically agents that learn through trial and error—can bridge the gap between theoretical stability and real-world reliability.

Methods

The authors conduct a comprehensive review of recent literature, focusing on the transition from classical control to Deep Reinforcement Learning (DRL). Key methodologies discussed include:

  • Reinforcement Learning Frameworks: Utilizing the Markov Decision Process (MDP) where an agent interacts with an environment to maximize a "reward signal."
  • Advanced Algorithms: The paper highlights specific implementations of TD3 (Twin-Delayed Deep Deterministic policy gradient), DDPG (Deep Deterministic Policy Gradient), and PPO (Proximal Policy Optimization).
  • Interactive RL (IRL): A novel approach where human trainers provide feedback to the AI, helping it learn faster in "sparse reward" environments (where the robot might wander for a long time before finding a goal).
  • Curriculum Learning: Training the AI on simple tasks first (like straight-line following) before introducing complexities like obstacles and heavy currents.
  • Hybrid Approaches: Combining DRL with traditional smoothing algorithms like NURBS (Non-Uniform Rational B-splines) to ensure the paths generated are physically feasible for a bulky underwater vehicle.

Datasets

As this is a review paper focused on control and robotics, it does not use static image datasets. Instead, it evaluates research performed in high-fidelity physics simulators:

  • Gazebo: A popular open-source robotics simulator.
  • UUV Simulator: A specialized package for Gazebo tailored for underwater environments.
  • Custom 3D Environments: Various studies cited used simulated "cylindrical helix" paths and "eight-shaped" trajectories to stress-test the algorithms.

Key Insights and Ideas

  • The "Black Box" Trade-off: While DRL agents outperform traditional controllers in complex paths (like "eight-shaped" curves), they operate as "black boxes." This lack of explainability is a major hurdle for safety-critical military or industrial missions.
  • Path Following vs. Path Planning: The paper clarifies a vital distinction. Path Following is about sticking to a pre-defined line without a strict schedule. Path Planning is the higher-level "brain" work of finding the optimal route from A to B while dodging moving ships or swimmers.
  • Human-in-the-loop Efficiency: One of the most interesting insights is that "Interactive RL" allows AUVs to converge to a solution much faster than pure environmental rewards alone, effectively "downloading" human intuition into the controller.
  • Robustness to Currents: AI-aided systems (like the improved DDQN) showed a superior ability to maintain a path even when hit by constant ocean current disturbances that would knock traditional systems off course.

Key Results

  • Complexity Handling: In tests involving closed-curve and eight-shaped paths, traditional PID controllers showed "obvious degradation" and overshoots, while DRL agents (specifically TD3-IMP) maintained high precision.
  • Training Speed: One study (Fang et al.) demonstrated that an AUV could be "mission-ready" after just 80 episodes of training (approx. 1.75 hours on an Intel i7 system), a significant improvement in sample efficiency.
  • 6-DoF Mastery: Recent DRL models have successfully achieved full six-degree-of-freedom control, allowing AUVs to manage posture (pitch, roll, yaw) and position simultaneously in 3D space.

Limitations and Future Work

Despite the optimism, the authors and the research they review point out several "anchors" holding the field back:

  • Explainability: There is a "limited acceptability" for AI in high-stakes environments because we can't always predict why the AI made a specific steering choice.
  • Sample Efficiency: While improving, many DRL models still require thousands of simulated "crashes" before they become proficient, which is impossible to do with expensive, real-world hardware.
  • The Path Planning Gap: Most current research focuses on following a path. The authors note that real-time path planning in dynamic environments (avoiding moving obstacles in a busy harbor) remains a "great difficulty" and a primary frontier for future research.

Reference: Sirelkhatim, A. A., Ali, M. O., & Peng, B. (2025). The Contributions of AI Usage Over AUVs’ Path-Following and Path-Planning. International Journal of Computer Applications, 187(28).

Summary updated

Abstract

The increasing of interesting in exploration of the underwater environment unknown and complexity, accentuated the need of the underwater vehicles with trusted robust control systems.Over a few pasts dedicates, much of researchers and scholars shown a huge racing for designing and implementing navigation systems supporting autonomy for underwater vehicles.This paper will explore the basic concepts of Autonomous Underwater Vehicle (AUV) control systems and terms.Two different criteria of algorithms for AUVs' path trajectory will discussed and explained (Path Following and Path Planning).Also, some path trajectory algorithms which have been designed with aiding of AI techniques are discussed through this research work; Where the study shows the similarities and differences between different types, and then assesses the benefits gained from the use of AI technology.
BibTeX
@article{sirelkhatim2025contributions,
  author = {Abeer Ali Sirelkhatim and Muhammad Ali and Bei Peng},
  title = {{The Contributions of AI Usage Over AUVs’ Path-Following and Path-Planning}},
  journal = {International Journal of Computer Applications},
  year = {2025},
  doi = {10.5120/ijca2025925489},
}
Figure