Fast and Efficient Locomotion via Learned Gait Transitions

Yuxiang Yang, Tingnan Zhang, Erwin Coumans, Jie Tan, Byron Boots

Abstract

We focus on the problem of developing energy efficient controllers for quadrupedal robots. Animals can actively switch gaits at different speeds to lower their energy consumption. In this paper, we devise a hierarchical learning framework, in which distinctive locomotion gaits and natural gait transitions emerge automatically with a simple reward of energy minimization. We use evolutionary strategies (ES) to train a high-level gait policy that specifies gait patterns of each foot, while the low-level convex MPC controller optimizes the motor commands so that the robot can walk at a desired velocity using that gait pattern. We test our learning framework on a quadruped robot and demonstrate automatic gait transitions, from walking to trotting and to fly-trotting, as the robot increases its speed. We show that the learned hierarchical controller consumes much less energy across a wide range of locomotion speed than baseline controllers.

Introduction

In nature, animals exhibit remarkable energy efficiency by dynamically switching gaits—walking at low speeds, trotting at medium speeds, and galloping at high speeds. Replicating this fluidity in robotics is challenging. Traditional controllers often rely on rigid, hand-engineered gait timings that lack adaptability. Conversely, end-to-end Reinforcement Learning (RL) frequently fails to bridge the "sim-to-real" gap, where simulated successes do not translate to physical hardware due to unmodeled dynamics and safety constraints. This work introduces a framework that enables robots to learn energy-efficient gaits and natural transitions automatically, maintaining the robustness required for real-world deployment.

Hierarchical Control Architecture

The proposed system utilizes a hierarchical framework that decouples gait timing from motor execution.

  1. High-Level Gait Policy (RL): A neural network serves as a gait generator. Rather than commanding torques directly, it outputs five parameters: stepping frequency, swing-to-stance ratio, and phase offsets for each leg. This policy is trained via Evolutionary Strategies (CMA-ES), optimizing for a reward that balances velocity tracking with energy minimization (Cost of Transport).
  2. Low-Level Convex MPC: This layer receives gait parameters and the robot's state to compute optimal ground reaction forces. It manages the fundamental physics of balancing and stable foot placement using a simplified centroidal dynamics model.
Figure 1: The hierarchical control architecture combining a high-level gait policy with a low-level convex MPC controller.

By employing Evolutionary Strategies (ES) instead of gradient-based RL, the low-level MPC can be treated as a black box, which proved more robust for this architecture.

Experimental Results

The framework was evaluated using the PyBullet physics engine for simulation and the Unitree A1 quadruped for hardware validation.

Emergent Gait Transitions

A key finding is that a simple energy minimization reward is sufficient for complex gaits to emerge. As the robot increases its speed, it automatically transitions from walking (< 0.9 m/s) to trotting (0.9–1.8 m/s) and finally to "fly-trotting" (> 1.8 m/s). Interestingly, the robot did not discover galloping; analysis revealed that for this specific morphology, galloping was 30% less efficient than trotting.

Energy Efficiency and Robustness

The learned policy consistently outperformed hand-tuned baseline controllers in terms of energy efficiency.

Figure 4: Comparison of Cost of Transport (CoT) across different velocities. The learned policy achieves lower energy consumption than standard gait baselines.

The robot reached speeds of 2.5 m/s (approximately 5 body lengths per second) and demonstrated "zero-shot" transfer to real-world environments, successfully navigating grass, carpet, and 8cm-high steps without additional fine-tuning.

Discussion and Future Work

The hierarchical approach acts as a bridge between high-level behavioral learning and low-level physical constraints. The MPC provides a safety net that allows the RL policy to be simpler and more transferable. Future work will explore expanding the high-level policy to control foot placement and base pose, potentially moving toward a bi-level optimization for tighter integration. The results suggest that future robotic designs should be co-optimized with their controllers to achieve animal-like agility and efficiency.

Summary updated

Abstract

We focus on the problem of developing energy efficient controllers for quadrupedal robots. Animals can actively switch gaits at different speeds to lower their energy consumption. In this paper, we devise a hierarchical learning framework, in which distinctive locomotion gaits and natural gait transitions emerge automatically with a simple reward of energy minimization. We use evolutionary strategies (ES) to train a high-level gait policy that specifies gait patterns of each foot, while the low-level convex MPC controller optimizes the motor commands so that the robot can walk at a desired velocity using that gait pattern. We test our learning framework on a quadruped robot and demonstrate automatic gait transitions, from walking to trotting and to fly-trotting, as the robot increases its speed. We show that the learned hierarchical controller consumes much less energy across a wide range of locomotion speed than baseline controllers.
BibTeX
@inproceedings{yang2021fast,
  author = {Yuxiang Yang and Tingnan Zhang and Erwin Coumans and Jie Tan and Byron Boots},
  title = {{Fast and Efficient Locomotion via Learned Gait Transitions}},
  booktitle = {CoRL},
  year = {2021},
  url = {https://openreview.net/forum?id=vm8Hr9YJHZ1},
}
Figure