TL;DR
FishGym is a high-performance, GPU-accelerated simulation framework designed to bridge the gap between simple empirical models and computationally expensive fluid dynamics for underwater robot learning. By implementing a localized Lattice Boltzmann Method (LBM) in a moving frame of reference, it enables reinforcement learning agents to discover complex, energy-efficient swimming behaviors—like "wake-riding" in schooling—that are impossible to capture in traditional simplified simulators.
What Problem Does This Paper Address?
Training intelligent underwater robots is notoriously difficult. Real-world testing is expensive and risky, while existing simulators often fall into two extremes:
- Empirical Models: These use simplified drag formulas that are fast but fail to capture complex fluid-structure interactions (FSI), such as vortices and turbulence.
- High-Fidelity CFD: Traditional Navier-Stokes solvers are physically accurate but prohibitively slow for Reinforcement Learning (RL), which requires millions of simulation steps.
Furthermore, simulating a fish swimming across a large distance usually requires a massive computational domain. The authors address this "efficiency vs. accuracy" trade-off by creating a platform that is fast enough for RL but detailed enough to model the subtle fluid forces that real fish exploit.
Methods
The FishGym framework consists of three integrated modules:
- Robot Modeling: Robots are represented as articulated rigid bodies with "skins" applied via linear blend skinning. This allows for arbitrary fish topologies (e.g., koi, eels, or flatfish).
- Localized Fluid Simulation: This is the core innovation. Instead of simulating a giant tank, FishGym simulates a small "local" fluid domain that moves with the fish. To maintain physical accuracy, the authors reformulated the Navier-Stokes equations for an accelerating frame of reference, adding virtual forces (Coriolis, centrifugal, etc.) to the fluid solver.
- GPU-Optimized Solver: They utilize a Lattice Boltzmann Method (LBM) coupled with an Immersed Boundary (IB) method. This setup is highly parallelizable on GPUs, achieving near real-time performance.
- Reinforcement Learning: The framework uses the Soft Actor-Critic (SAC) algorithm via the
stable-baselines3library to train control policies for specific tasks.
Datasets
While the paper doesn't use a static external dataset, it introduces a suite of four benchmark tasks for underwater RL:
- Cruising: Reaching a distant target.
- Pose Control: Executing precise maneuvers like a U-turn.
- Two-fish Schooling: Following a leader fish as closely as possible.
- Path Following: Tracking an arbitrary 3D trajectory.
The authors also provide a GitHub repository (fish-gym) containing the simulation environment and trained models.
Key Insights and Ideas
- The "Vortex-Riding" Discovery: In the two-fish schooling task, the RL agent discovered a behavior where the follower fish utilizes the "wake vortex ring" generated by the leader to preserve energy. This emergent behavior was only possible because of the high-fidelity fluid simulation; agents trained on empirical models simply followed in a straight line, missing the energy-saving physics.
- Local Frame Efficiency: By moving the simulation box with the agent, the framework can simulate "infinite" swimming distances with a constant, low memory footprint.
- Physical Consistency: The authors demonstrated that the simulator correctly responds to fluid density changes. For example, in lower-density fluids, the fish learned to use larger tail-swing amplitudes to generate sufficient propulsion, matching physical expectations.
Key Results
- Performance: The solver simulates one physical second in approximately 3.5 to 4 seconds on an NVIDIA TitanXp GPU.
- Training Speed: Control policies typically converge within 6 to 10 hours (roughly 1,000 episodes).
- Accuracy: In path-following tasks, the fish achieved a mean deviation as low as 0.03m over a 7-meter path. In contrast, an un-tuned empirical model showed significant "drifting" during turns with errors up to 0.5m.
- Versatility: The framework successfully trained diverse morphologies, including a branching skeleton for a flatfish and a long concatenated skeleton for an eel.
Limitations and Future Work
- Sim-to-Real Gap: The fish models are currently "reduced" (skeletal structures with skin), which may not perfectly capture the soft-body dynamics of real bionic robots.
- Environmental Complexity: The current local-domain approach struggles with complex external environments, such as large-scale ambient currents or obstacles that exist outside the moving "box."
- Grid Resolution: While fast, the grid resolution is a bottleneck for accuracy. Future work aims to implement more efficient multi-scale or adaptive grid methods to capture even finer turbulent details without sacrificing speed.
Summary updated