Learning Whole-Body Control for a Salamander Robot

Mengze Tian, Qiyuan Fu, Chuanfang Ning, Javier Jia Jie Pey, Auke Jan Ijspeert

Abstract

Amphibious legged robots inspired by salamanders are promising in applications in complex amphibious environments. However, despite the significant success of training controllers that achieve diverse locomotion behaviors in conventional quadrupedal robots, most salamander robots relied on central-pattern-generator (CPG)-based and model-based coordination strategies for locomotion control. Learning unified joint-level whole-body control that reliably transfers from simulation to highly articulated physical salamander robots remains relatively underexplored. In addition, few legged robots have tried learning-based controllers in amphibious environments. In this work, we employ Reinforcement Learning to map proprioceptive observations and commanded velocities to joint-level actions, allowing coordinated locomotor behaviors to emerge. To deploy these policies on hardware, we adopt a system-level real-to-sim matching and sim-to-real transfer strategy. The learned controller achieves stable and coordinated walking on both flat and uneven terrains in the real world. Beyond terrestrial locomotion, the framework enables transitions between walking and swimming in simulation, highlighting a phenomenon of interest for understanding locomotion across distinct physical modes.

Introduction

Salamanders are biological marvels of amphibious locomotion, utilizing an articulated spine and four limbs to navigate both land and water. Replicating this versatility in robotics is challenging due to the high number of degrees of freedom (DoF) and the complex dynamics involved in transitioning between contact-driven terrestrial movement and fluid-driven aquatic movement. Traditionally, these robots have relied on Central Pattern Generators (CPGs)—biologically inspired oscillators—which are difficult to scale for omnidirectional movement and often struggle with the hybrid dynamics of amphibious environments. Furthermore, the "sim-to-real gap" is particularly pronounced in long, serial-body robots where small errors in joint alignment or motor behavior accumulate along the kinematic chain.

Methods

This research frames locomotion as a joint-space control problem solved via Proximal Policy Optimization (PPO). The framework maps proprioceptive observations and commanded velocities directly to joint-level actions.

Key Technical Components

  • Residual Action Space: The policy outputs incremental changes to a fixed nominal posture rather than absolute joint angles, ensuring safer and more stable exploration during training.
  • Sim-to-Real Alignment: To bridge the gap to reality, the team implemented a three-layer alignment strategy: injecting sensor noise, modeling non-linear motor torque/speed limits ("servo envelopes"), and introducing a "Backlash Model." The latter involves adding passive rotational joints in simulation to mimic the mechanical "play" found in the physical robot's connections.
  • Hybrid Dynamics Simulation: The MuJoCo environment was enhanced with a simplified hydrodynamic model (buoyancy and drag) to simulate aquatic environments alongside standard rigid-body physics for land.
  • Weak Rhythmic Prior: A "phase" input and a reward term encourage rhythmic foot height profiles, allowing the robot to learn ground clearance without hard-coding specific gaits.
The simulation framework enables the emergence of body-driven undulatory swimming behaviors and seamless transitions between land and water.

Results and Insights

The learned controller achieved stable, omnidirectional walking across various real-world environments. On flat ground, the robot reached speeds of 0.23 m/s. It also demonstrated significant robustness on rugged terrain with 4cm height variations and 25° slopes, maintaining a speed of 0.17 m/s.

The salamander robot demonstrates robust locomotion across diverse real-world terrains, including gravel, grass, and pavement, as well as complex simulation environments.

Key Findings

  • Backlash Modeling is Critical: Modeling mechanical looseness was essential for accurate foot clearance in the real world. Without it, the simulation overestimated the robot's ability to clear obstacles.
  • Emergent Biology: In aquatic simulations, the policy naturally shifted from limb-driven stepping to "body-driven undulatory swimming." This traveling wave behavior in the spine emerged as an efficient solution to fluid dynamics without being explicitly programmed.
  • Velocity Inference: The robot succeeded in omnidirectional movement without explicit linear velocity sensing, instead using a short history of past actions to infer its own dynamics.

Discussion and Future Work

While the terrestrial walking was successfully validated on hardware, the amphibious transitions remain simulation-based. Future work will address the physical challenges of aquatic deployment, such as waterproofing and real-world fluid dynamics. Additionally, the researchers aim to improve onboard state estimation to reduce tracking errors between commanded and actual velocities.

Summary updated

Abstract

Amphibious legged robots inspired by salamanders are promising in applications in complex amphibious environments. However, despite the significant success of training controllers that achieve diverse locomotion behaviors in conventional quadrupedal robots, most salamander robots relied on central-pattern-generator (CPG)-based and model-based coordination strategies for locomotion control. Learning unified joint-level whole-body control that reliably transfers from simulation to highly articulated physical salamander robots remains relatively underexplored. In addition, few legged robots have tried learning-based controllers in amphibious environments. In this work, we employ Reinforcement Learning to map proprioceptive observations and commanded velocities to joint-level actions, allowing coordinated locomotor behaviors to emerge. To deploy these policies on hardware, we adopt a system-level real-to-sim matching and sim-to-real transfer strategy. The learned controller achieves stable and coordinated walking on both flat and uneven terrains in the real world. Beyond terrestrial locomotion, the framework enables transitions between walking and swimming in simulation, highlighting a phenomenon of interest for understanding locomotion across distinct physical modes.
BibTeX
@misc{tian2026learning,
  author = {Mengze Tian and Qiyuan Fu and Chuanfang Ning and Javier Jia Jie Pey and Auke Jan Ijspeert},
  title = {{Learning Whole-Body Control for a Salamander Robot}},
  year = {2026},
  eprint = {2603.16683},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.16683},
}
Figure