Toward the Intelligent, Safe Exploration of a Biomimetic Underwater Robot: Modeling, Planning, and Control

Yu Wang, Jian Wang, Lianyi Yu, Shihan Kong, Junzhi YuBiomimetics 2024 · DOI

TL;DR

This paper presents an intelligent framework for safe underwater exploration using a biomimetic robotic fish. It combines a novel obstacle detection and collision model with a deep reinforcement learning-based motion planner and a backstepping yaw controller, enabling the robot to navigate complex, obstacle-filled environments autonomously.

What Problem Does This Paper Address?

Exploring the ocean's depths is crucial for resource discovery and scientific understanding, but it's fraught with challenges. Underwater environments are often complex, featuring dense coral reefs, uneven terrain, and numerous obstacles. Traditional propeller-driven underwater robots struggle with maneuverability and can disturb delicate ecosystems. Biomimetic robots, inspired by marine life, offer a promising alternative due to their agility and low environmental impact. However, achieving safe and intelligent autonomous navigation in these cluttered environments remains a significant hurdle. This paper aims to bridge this gap by developing a comprehensive system for a biomimetic robotic fish that can intelligently plan its path and control its movement while actively avoiding collisions.

Methods

The proposed framework is built upon three core components:

  • Obstacle Detection and Collision Model: The researchers equip a biomimetic robotic fish with three distance sensors (forward, left, and right). They develop models for two common obstacle types: rectangular and circular. The system calculates the distance to these obstacles by modeling the sensors as line segments and performing geometric intersection calculations with the obstacle boundaries. This provides real-time environmental feedback.
  • Deep Reinforcement Learning (DRL) Motion Planner: To achieve intelligent motion planning, a Deep Deterministic Policy Gradient (DDPG) algorithm is employed. The DDPG agent learns to output a target yaw angle increment based on a state space that includes the robot's distance to the target, heading error, and the distances from the three sensors to the nearest obstacles. The action space is defined as the yaw angle increment, and a carefully designed reward function encourages shortest path, correct heading, smooth motion, obstacle avoidance, and reaching the goal. The paper also investigates different training strategies, including static and dynamic obstacle environments, to optimize the planner's performance.
  • Backstepping-Based Yaw Controller: To ensure smooth and stable execution of the planned motion, a backstepping control method is used for yaw control. This controller takes the target yaw angle from the DDPG planner and generates the necessary control commands. Crucially, a sigmoid function-based smoothing method is incorporated to handle potential discontinuities in the planner's output, preventing abrupt changes in the target heading and improving overall system stability.

Datasets

Not discussed in the paper. The paper focuses on a simulated environment for training and evaluation.

Key Insights and Ideas

  • Biomimetic Advantage: The paper highlights how biomimetic robots, with their inherent maneuverability, are well-suited for navigating complex underwater environments where traditional robots struggle.
  • Integrated Framework: The strength of the approach lies in the seamless integration of obstacle detection, intelligent motion planning (DRL), and robust control (backstepping), creating a holistic solution for autonomous exploration.
  • DRL for Complex Environments: DDPG proves effective in learning complex navigation policies in environments with multiple, potentially dynamic, obstacles, demonstrating the power of reinforcement learning for such tasks.
  • Smooth Control for Stability: The use of a sigmoid function to smooth the DRL planner's output is a critical insight, as it significantly improves the stability of the yaw controller and prevents jerky movements, which is vital for underwater operations.
  • Training Environment Matters: The study shows that training the DRL agent in a static obstacle environment leads to better performance compared to dynamic environments, suggesting that a stable, well-defined optimal path can be more effectively learned.

Key Results

The simulations demonstrate that the proposed framework successfully enables the biomimetic robotic fish to:

  • Intelligently plan paths: The DDPG planner effectively navigates through environments with both rectangular and circular obstacles.
  • Safely avoid collisions: The integrated obstacle detection and collision model, coupled with the DRL planner, ensures the robot avoids hitting obstacles.
  • Achieve stable yaw control: The backstepping controller, enhanced by the sigmoid smoothing, provides smooth and accurate yaw angle tracking, even when encountering sudden changes in the planned trajectory.
  • Converge rapidly: The DRL planner shows rapid convergence during training, indicating efficient learning.
  • Perform well in complex scenarios: The robot successfully navigates through multiple stages of complex obstacle arrangements, demonstrating robustness.

Limitations and Future Work

The authors acknowledge several limitations and propose future research directions:

  • Robot Shape Modeling: The current model treats the robotic fish as a point, neglecting its actual shape and size. Future work will involve incorporating the robot's physical dimensions into the models for more realistic simulations and control.
  • DRL Action Space Issues: DDPG can sometimes get stuck in extreme action regions or learn strategies at the edges of the action space. The authors suggest exploring new network architectures or incorporating other DRL algorithms to mitigate this.
  • Prototype Development and Aquatic Experiments: The current work is based on simulations. The next crucial step is to develop a physical prototype of the robotic fish and conduct real-world aquatic experiments to validate the proposed methods.
  • Underwater Stereoscopic Vision: To gather richer environmental information, the researchers plan to integrate underwater stereoscopic vision for more detailed obstacle identification.
  • Vision-Based Navigation: Developing a vision-based navigation algorithm is identified as another promising area for future research.

Summary updated

Abstract

Safe, underwater exploration in the ocean is a challenging task due to the complex environment, which often contains areas with dense coral reefs, uneven terrain, or many obstacles. To address this issue, an intelligent underwater exploration framework of a biomimetic robot is proposed in this paper, including an obstacle avoidance model, motion planner, and yaw controller. Firstly, with the aid of the onboard distance sensors in robotic fish, the obstacle detection model is established. On this basis, two types of obstacles, i.e., rectangular and circular, are considered, followed by the obstacle collision model's construction. Secondly, a deep reinforcement learning method is adopted to plan the plane motion, and the performances of different training setups are investigated. Thirdly, a backstepping method is applied to derive the yaw control law, in which a sigmoid function-based transition method is employed to smooth the planning output. Finally, a series of simulations are carried out to verify the effectiveness of the proposed method. The obtained results indicate that the biomimetic robot can not only achieve intelligent motion planning but also accomplish yaw control with obstacle avoidance, offering a valuable solution for underwater operation in the ocean.
BibTeX
@inproceedings{wang2024toward,
  author = {Yu Wang and Jian Wang and Lianyi Yu and Shihan Kong and Junzhi Yu},
  title = {{Toward the Intelligent, Safe Exploration of a Biomimetic Underwater Robot: Modeling, Planning, and Control}},
  booktitle = {Biomimetics},
  year = {2024},
  doi = {10.3390/biomimetics9030126},
}
Figure