FishGym: A High-Performance Physics-based Simulation Framework for Underwater Robot Learning

Wenji Liu, Kai Bai, Xuming He, Shuran Song, Changxi Zheng, Xiaopei Liu

TL;DR

FishGym is a high-performance, GPU-accelerated simulation framework designed to bridge the gap between simple empirical models and computationally expensive fluid dynamics for underwater robot learning. By implementing a localized Lattice Boltzmann Method (LBM) in a moving frame of reference, it enables reinforcement learning agents to discover complex, energy-efficient swimming behaviors—like "wake-riding" in schooling—that are impossible to capture in traditional simplified simulators.

What Problem Does This Paper Address?

Training intelligent underwater robots is notoriously difficult. Real-world testing is expensive and risky, while existing simulators often fall into two extremes:

  1. Empirical Models: These use simplified drag formulas that are fast but fail to capture complex fluid-structure interactions (FSI), such as vortices and turbulence.
  2. High-Fidelity CFD: Traditional Navier-Stokes solvers are physically accurate but prohibitively slow for Reinforcement Learning (RL), which requires millions of simulation steps.

Furthermore, simulating a fish swimming across a large distance usually requires a massive computational domain. The authors address this "efficiency vs. accuracy" trade-off by creating a platform that is fast enough for RL but detailed enough to model the subtle fluid forces that real fish exploit.

Methods

The FishGym framework consists of three integrated modules:

  • Robot Modeling: Robots are represented as articulated rigid bodies with "skins" applied via linear blend skinning. This allows for arbitrary fish topologies (e.g., koi, eels, or flatfish).
  • Localized Fluid Simulation: This is the core innovation. Instead of simulating a giant tank, FishGym simulates a small "local" fluid domain that moves with the fish. To maintain physical accuracy, the authors reformulated the Navier-Stokes equations for an accelerating frame of reference, adding virtual forces (Coriolis, centrifugal, etc.) to the fluid solver.
  • GPU-Optimized Solver: They utilize a Lattice Boltzmann Method (LBM) coupled with an Immersed Boundary (IB) method. This setup is highly parallelizable on GPUs, achieving near real-time performance.
  • Reinforcement Learning: The framework uses the Soft Actor-Critic (SAC) algorithm via the stable-baselines3 library to train control policies for specific tasks.

Datasets

While the paper doesn't use a static external dataset, it introduces a suite of four benchmark tasks for underwater RL:

  1. Cruising: Reaching a distant target.
  2. Pose Control: Executing precise maneuvers like a U-turn.
  3. Two-fish Schooling: Following a leader fish as closely as possible.
  4. Path Following: Tracking an arbitrary 3D trajectory.

The authors also provide a GitHub repository (fish-gym) containing the simulation environment and trained models.

Key Insights and Ideas

  • The "Vortex-Riding" Discovery: In the two-fish schooling task, the RL agent discovered a behavior where the follower fish utilizes the "wake vortex ring" generated by the leader to preserve energy. This emergent behavior was only possible because of the high-fidelity fluid simulation; agents trained on empirical models simply followed in a straight line, missing the energy-saving physics.
  • Local Frame Efficiency: By moving the simulation box with the agent, the framework can simulate "infinite" swimming distances with a constant, low memory footprint.
  • Physical Consistency: The authors demonstrated that the simulator correctly responds to fluid density changes. For example, in lower-density fluids, the fish learned to use larger tail-swing amplitudes to generate sufficient propulsion, matching physical expectations.

Key Results

  • Performance: The solver simulates one physical second in approximately 3.5 to 4 seconds on an NVIDIA TitanXp GPU.
  • Training Speed: Control policies typically converge within 6 to 10 hours (roughly 1,000 episodes).
  • Accuracy: In path-following tasks, the fish achieved a mean deviation as low as 0.03m over a 7-meter path. In contrast, an un-tuned empirical model showed significant "drifting" during turns with errors up to 0.5m.
  • Versatility: The framework successfully trained diverse morphologies, including a branching skeleton for a flatfish and a long concatenated skeleton for an eel.

Limitations and Future Work

  • Sim-to-Real Gap: The fish models are currently "reduced" (skeletal structures with skin), which may not perfectly capture the soft-body dynamics of real bionic robots.
  • Environmental Complexity: The current local-domain approach struggles with complex external environments, such as large-scale ambient currents or obstacles that exist outside the moving "box."
  • Grid Resolution: While fast, the grid resolution is a bottleneck for accuracy. Future work aims to implement more efficient multi-scale or adaptive grid methods to capture even finer turbulent details without sacrificing speed.

Summary updated

Abstract

Bionic underwater robots have demonstrated their superiority in many applications. Yet, training their intelligence for a variety of tasks that mimic the behavior of underwater creatures poses a number of challenges in practice, mainly due to lack of a large amount of available training data as well as the high cost in real physical environment. Alternatively, simulation has been considered as a viable and important tool for acquiring datasets in different environments, but it mostly targeted rigid and soft body systems. There is currently dearth of work for more complex fluid systems interacting with immersed solids that can be efficiently and accurately simulated for robot training purposes. In this paper, we propose a new platform called “FishGym”, which can be used to train fish-like underwater robots. The framework consists of a robotic fish modeling module using articulated body with skinning, a GPU-based high-performance localized two-way coupled fluid-structure interaction simulation module that handles both finite and infinitely large domains, as well as a reinforcement learning module. We leveraged existing training methods with adaptations to underwater fish-like robots and obtained learned control policies for multiple benchmark tasks. The training results are demonstrated with reasonable motion trajectories, with comparisons and analyses to empirical models as well as known real fish swimming behaviors to highlight the advantages of the proposed platform.
BibTeX
@inproceedings{liu2022fishgym,
  author = {Wenji Liu and Kai Bai and Xuming He and Shuran Song and Changxi Zheng and Xiaopei Liu},
  title = {{FishGym: A High-Performance Physics-based Simulation Framework for Underwater Robot Learning}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2022},
  url = {https://openreview.net/forum?id=LzeCBSQUMX},
}

Loading related content...

Figure