FishGym: A High-Performance Physics-based Simulation Framework for Underwater Robot Learning

Wenji Liu, Kai Bai, Xuming He, Shuran Song, Changxi Zheng, Xiaopei Liu

Abstract

Bionic underwater robots have demonstrated their superiority in many applications. Yet, training their intelligence for a variety of tasks that mimic the behavior of underwater creatures poses a number of challenges in practice, mainly due to lack of a large amount of available training data as well as the high cost in real physical environment. Alternatively, simulation has been considered as a viable and important tool for acquiring datasets in different environments, but it mostly targeted rigid and soft body systems. There is currently dearth of work for more complex fluid systems interacting with immersed solids that can be efficiently and accurately simulated for robot training purposes. In this paper, we propose a new platform called "FishGym", which can be used to train fish-like underwater robots. The framework consists of a robotic fish modeling module using articulated body with skinning, a GPU-based high-performance localized two-way coupled fluid-structure interaction simulation module that handles both finite and infinitely large domains, as well as a reinforcement learning module. We leveraged existing training methods with adaptations to underwater fish-like robots and obtained learned control policies for multiple benchmark tasks. The training results are demonstrated with reasonable motion trajectories, with comparisons and analyses to empirical models as well as known real fish swimming behaviors to highlight the advantages of the proposed platform.

Introduction

Training intelligent underwater robots is notoriously difficult. Real-world testing is expensive and risky, while existing simulators often fall into two extremes: simplified empirical models and high-fidelity Computational Fluid Dynamics (CFD). Empirical models use simplified drag formulas that are fast but fail to capture complex fluid-structure interactions (FSI), such as vortices and turbulence. Conversely, traditional Navier-Stokes solvers are physically accurate but prohibitively slow for Reinforcement Learning (RL), which requires millions of simulation steps.

Furthermore, simulating a fish swimming across a large distance usually requires a massive computational domain. FishGym addresses this "efficiency vs. accuracy" trade-off by creating a platform that is fast enough for RL but detailed enough to model the subtle fluid forces that real fish exploit.

Methodology

The FishGym framework consists of three integrated modules designed for high performance and physical realism:

  1. Robot Modeling: Robots are represented as articulated rigid bodies with "skins" applied via linear blend skinning. This allows for arbitrary fish topologies, ranging from koi to eels.
  2. Localized Fluid Simulation: Instead of simulating a giant tank, FishGym simulates a small "local" fluid domain that moves with the fish. To maintain physical accuracy, the authors reformulated the Navier-Stokes equations for an accelerating frame of reference, adding virtual forces (Coriolis, centrifugal, etc.) to the fluid solver. This allows for "infinite" swimming distances with a constant, low memory footprint.
  3. GPU-Optimized Solver: The system utilizes a Lattice Boltzmann Method (LBM) coupled with an Immersed Boundary (IB) method. This setup is highly parallelizable on GPUs, achieving near real-time performance.
Articulated body modeling of a robotic fish using a skeletal structure with skinning.

For the learning component, the framework uses the Soft Actor-Critic (SAC) algorithm to train control policies for specific tasks.

Benchmark Tasks

FishGym introduces a suite of four benchmark tasks for underwater RL:

  • Cruising: Reaching a distant target.
  • Pose Control: Executing precise maneuvers like a U-turn.
  • Two-fish Schooling: Following a leader fish as closely as possible.
  • Path Following: Tracking an arbitrary 3D trajectory.

Results and Discussion

The framework demonstrates significant performance and qualitative advantages over traditional methods. The solver simulates one physical second in approximately 3.5 to 4 seconds on an NVIDIA TitanXp GPU, with control policies typically converging within 6 to 10 hours.

The "Vortex-Riding" Discovery

A key insight emerged during the two-fish schooling task. The RL agent discovered a behavior where the follower fish utilizes the "wake vortex ring" generated by the leader to preserve energy. This emergent behavior was only possible because of the high-fidelity fluid simulation. In contrast, agents trained on empirical models simply followed in a straight line, missing the energy-saving physics entirely.

Comparison of schooling behavior: Empirical models (top) fail to capture fluid interactions, while FishGym (bottom) enables agents to exploit wake vortices.

Accuracy and Versatility

In path-following tasks, the fish achieved a mean deviation as low as 0.03m over a 7-meter path. An un-tuned empirical model showed significant "drifting" during turns with errors up to 0.5m. The framework also successfully trained diverse morphologies, including a branching skeleton for a flatfish and a long concatenated skeleton for an eel.

Conclusion and Future Work

While FishGym successfully bridges the gap between speed and accuracy, some limitations remain. The current fish models use reduced skeletal structures which may not perfectly capture the soft-body dynamics of real bionic robots. Additionally, the local-domain approach struggles with complex external environments, such as large-scale ambient currents or obstacles outside the moving simulation box. Future work aims to implement more efficient multi-scale or adaptive grid methods to capture even finer turbulent details without sacrificing the speed necessary for reinforcement learning.

Summary updated

Abstract

Bionic underwater robots have demonstrated their superiority in many applications. Yet, training their intelligence for a variety of tasks that mimic the behavior of underwater creatures poses a number of challenges in practice, mainly due to lack of a large amount of available training data as well as the high cost in real physical environment. Alternatively, simulation has been considered as a viable and important tool for acquiring datasets in different environments, but it mostly targeted rigid and soft body systems. There is currently dearth of work for more complex fluid systems interacting with immersed solids that can be efficiently and accurately simulated for robot training purposes. In this paper, we propose a new platform called "FishGym", which can be used to train fish-like underwater robots. The framework consists of a robotic fish modeling module using articulated body with skinning, a GPU-based high-performance localized two-way coupled fluid-structure interaction simulation module that handles both finite and infinitely large domains, as well as a reinforcement learning module. We leveraged existing training methods with adaptations to underwater fish-like robots and obtained learned control policies for multiple benchmark tasks. The training results are demonstrated with reasonable motion trajectories, with comparisons and analyses to empirical models as well as known real fish swimming behaviors to highlight the advantages of the proposed platform.
BibTeX
@inproceedings{liu2022fishgym,
  author = {Wenji Liu and Kai Bai and Xuming He and Shuran Song and Changxi Zheng and Xiaopei Liu},
  title = {{FishGym: A High-Performance Physics-based Simulation Framework for Underwater Robot Learning}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2022},
  url = {https://openreview.net/forum?id=LzeCBSQUMX},
}
Figure