Leveraging Symmetries in Gaits for Reinforcement Learning: A Case Study on Quadrupedal Gaits

Jiayu Ding, Xulin Chen, Garrett E. Katz, Zhenyu Gan

Abstract

Quadrupedal robots exhibit a wide range of viable gaits, but generating specific footfall sequences often requires laborious expert tuning of numerous variables, such as touch-down and lift-off events and holonomic constraints for each leg. This paper presents a unified reinforcement learning framework for generating versatile quadrupedal gaits by leveraging the intrinsic symmetries of dynamic legged systems. We propose a symmetry-guided reward function design that incorporates temporal, morphological, and time-reversal symmetries, streamlining gait design, accelerating learning, and enhancing robot agility. By focusing on preserved symmetries and natural dynamics, our approach eliminates the need for predefined trajectories or footfall sequences, enabling smooth transitions between diverse locomotion patterns such as trotting, bounding, half-bounding, and galloping. Implemented on the Unitree Go2 robot, our method demonstrates robust performance across a range of speeds in both simulations and hardware tests, significantly improving gait adaptability without extensive reward tuning or explicit foot placement control.

Introduction

While quadrupedal robots have achieved remarkable agility, enabling them to switch seamlessly between different locomotion patterns—or "gears"—remains a significant challenge. Traditional approaches typically fall into two categories: Trajectory Optimization/Model Predictive Control (MPC) and standard Reinforcement Learning (RL). Trajectory optimization often relies on fixed, hand-coded footfall sequences that can struggle with environmental uncertainty. Conversely, standard RL, while flexible, often necessitates extensive "reward engineering"—the tedious manual tuning of numerous variables to elicit natural-looking motion.

Overview of the Symmetry-Guided RL framework, illustrating the integration of gait parameters with temporal and morphological symmetries to produce diverse locomotion patterns like trotting and galloping.

This research addresses the lack of a unified framework capable of generating a full spectrum of gaits and transitioning between them on demand based on speed, mimicking the natural transitions observed in biological systems like horses shifting from a walk to a gallop.

Methodology: Symmetry-Guided RL Hierarchy

The proposed framework utilizes a symmetry-guided RL hierarchy. Rather than prescribing exact foot placements, the system provides a mathematical "nudge" based on the intrinsic physics of legged systems.

Gait Specification and Coupling

The framework employs a systematic parameterization of gaits based on phase offsets and duty factors. A key innovation is the coupling of the stride period ($T$) and duty factor ($\beta$) to the robot's forward velocity ($v_x$). This coupling is informed by the passive dynamics of Spring-Loaded Inverted Pendulum (SLIP) models, ensuring that gait timing remains natural relative to the commanded speed.

Symmetry-Enforced Rewards

The core of the methodology is the reward function ($R_{sym}$), which embeds three distinct types of symmetry:

  • Temporal Symmetry: This component penalizes foot movement during the stance phase and ground reaction forces during the swing phase, ensuring clear, periodic stepping cycles.
  • Morphological Symmetry: This encourages pairs of legs sharing the same phase offset to produce similar joint trajectories. This prevents "limping" and significantly improves inter-limb coordination.
  • Time-Reversal Symmetry: Enforced through phase mapping, this ensures that backward motion is a mirror image of forward motion, preventing the policy from developing biases toward specific legs when changing directions.
The complete training framework design, showing the flow from user commands and gait parametrization through the symmetry-guided reward function to policy deployment on the Go2 robot.

Training and Deployment

The policy was trained using Proximal Policy Optimization (PPO) within the NVIDIA Isaac Gym simulator, utilizing 2,048 parallel agents and randomized dynamics (mass, friction, etc.) to ensure robustness. The resulting Multi-Layer Perceptron (MLP) policy outputs joint targets for a PD controller running at 50 Hz on a Unitree Go2 robot.

Key Results and Insights

The inclusion of symmetry as a fundamental design principle yielded several critical advantages:

  • Dimensionality Reduction: Enforcing symmetry reduces the search space for the RL agent, as it does not need to independently learn movements for symmetric limbs.
  • Energy Efficiency: Morphological symmetry was found to be a major driver of efficiency. Policies enforcing this symmetry reduced the Cost of Transport (CoT) by 30–50% compared to variants without it. For example, the full policy achieved a CoT of 2.86 at 0.2 m/s, versus 4.15 without morphological constraints.
  • Seamless Transitions: The unified nature of the framework allows the robot to transition between gaits (e.g., from galloping to trotting) within a single step (approximately 0.5s) without specialized transition logic.
  • Hardware Robustness: In real-world tests, the Go2 robot successfully tracked velocities from -2.0 m/s to 2.0 m/s and maintained stability across diverse surfaces, including smooth concrete and uneven grass.

Conclusion and Future Work

This work demonstrates that leveraging intrinsic symmetries can significantly streamline the development of versatile robotic gaits. By embedding physical principles directly into the reward structure, the framework achieves high-fidelity gait reproduction and improved energy efficiency. Future research will focus on extending these capabilities to more complex terrains, such as stairs and steep slopes, and developing autonomous high-level policies for automatic gait selection based on environmental conditions.

Summary updated

Abstract

Quadrupedal robots exhibit a wide range of viable gaits, but generating specific footfall sequences often requires laborious expert tuning of numerous variables, such as touch-down and lift-off events and holonomic constraints for each leg. This paper presents a unified reinforcement learning framework for generating versatile quadrupedal gaits by leveraging the intrinsic symmetries of dynamic legged systems. We propose a symmetry-guided reward function design that incorporates temporal, morphological, and time-reversal symmetries, streamlining gait design, accelerating learning, and enhancing robot agility. By focusing on preserved symmetries and natural dynamics, our approach eliminates the need for predefined trajectories or footfall sequences, enabling smooth transitions between diverse locomotion patterns such as trotting, bounding, half-bounding, and galloping. Implemented on the Petoi Bittle robot, our method demonstrates robust performance across a range of speeds in both simulations and hardware tests, significantly improving gait adaptability without extensive reward tuning or explicit foot placement control. This work provides insights into dynamic locomotion strategies and underscores the crucial role of symmetries in robotic gait design.
BibTeX
@misc{ding2025leveraging,
  author = {Jiayu Ding and Xulin Chen and Garrett E. Katz and Zhenyu Gan},
  title = {{Leveraging Symmetries in Gaits for Reinforcement Learning: A Case Study on Quadrupedal Gaits}},
  year = {2025},
  eprint = {2510.10455},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.10455},
}
Figure