Gait in Eight: Efficient On-Robot Learning for Omnidirectional Quadruped Locomotion

Nico Bohlinger, Jonathan Kinzel, Daniel Palenicek, Lukasz Antczak, Jan Peters

Introduction

Deep Reinforcement Learning (DRL) has revolutionized quadruped locomotion, yet most state-of-the-art systems rely heavily on massive simulation-to-real (sim-to-real) transfer. While simulation provides a safe environment for millions of training steps, it often suffers from an "embodiment gap"—discrepancies in motor friction, battery levels, and structural wear that simulation cannot perfectly replicate. This research presents a framework that bypasses extensive simulation by enabling a quadruped robot to learn agile, omnidirectional locomotion directly on physical hardware in just 8 minutes of real-time training.

The Challenge of On-Robot Learning

Learning directly on a physical robot introduces two primary hurdles: sample efficiency and computational constraints. Unlike simulations, physical robots are subject to mechanical wear and tear; they cannot endure millions of exploratory steps. Furthermore, on-board computers typically lack the massive GPU clusters used for modern RL training. Previous attempts at on-robot learning often simplified the task to basic forward walking or utilized computationally intensive algorithms like DroQ, which require high update-to-data (UTD) ratios that tax hardware resources.

Methodology

The CrossQ Algorithm

At the core of this framework is CrossQ, a recent off-policy RL algorithm designed for high sample efficiency. Unlike standard algorithms such as Soft Actor-Critic (SAC) that utilize "target networks" to stabilize learning, CrossQ incorporates Batch Normalization within the critic network. This architectural choice allows the system to maintain a UTD ratio of 1—meaning it performs exactly one gradient update per environment interaction. This significantly reduces the computational load, making real-time training feasible on consumer-grade hardware like an M3 MacBook Pro.

Control Architectures

The researchers evaluated two distinct methods for translating AI commands into motor actions:

  1. Joint Target Prediction (JTP): The agent predicts raw joint offsets directly. This approach favors high-speed, aggressive movements and agility.
  2. Central Pattern Generators (CPG): The agent modifies a pre-defined rhythmic "trot" cycle. This architecture results in more stable, natural-looking gaits and is better suited for omnidirectional movement.

State Estimation and Filtering

To compensate for the MAB HoneyBadger robot's lack of foot-contact sensors, the team implemented a Kalman filter that fuses IMU data with leg odometry to estimate velocity. To ensure responsiveness, they utilized a One-Euro filter, which adapts its smoothing parameters based on velocity—providing stability at low speeds without sacrificing responsiveness during high-speed maneuvers.

Experimental Results

Training Efficiency and Speed

The framework achieved high performance in just 1 minute of simulation and 8 minutes of real-world training. This represents a 5x improvement in training speed over DroQ, the previous benchmark for on-robot learning. Using the JTP architecture, the robot reached forward speeds of 0.85 m/s on a smooth indoor surface.

Agility vs. Stability

The study revealed a clear trade-off between the two control architectures. While JTP was faster, it was prone to falling and struggled with complex omnidirectional commands. Conversely, the CPG architecture was slower (peaking at 0.33 m/s) but demonstrated superior stability. CPG-based agents mastered 360-degree movement in approximately 33 minutes of training without falling, successfully navigating both indoor floors and outdoor cobblestones.

Hardware Independence

A significant finding was the ability to train these models using a standard laptop connected via Ethernet. This proves that high-end, dedicated server GPUs are not a prerequisite for sophisticated on-robot reinforcement learning, lowering the barrier to entry for robotics research.

Limitations and Future Work

Despite the success of the "Gait in Eight" framework, limitations remain in velocity estimation. Without dedicated foot sensors, the robot assumes constant ground contact during odometry, leading to a systematic underestimation of speed. This underestimation currently acts as a ceiling for gait quality. Future research will focus on integrating visual-inertial odometry to provide the agent with a more accurate sense of its surroundings and its own velocity, as well as automating the curriculum required for JTP to master omnidirectional tasks.

Summary updated

BibTeX
@inproceedings{bohlinger2025gait,
  author = {Nico Bohlinger and Jonathan Kinzel and Daniel Palenicek and Lukasz Antczak and Jan Peters},
  title = {{Gait in Eight: Efficient On-Robot Learning for Omnidirectional Quadruped Locomotion}},
  booktitle = {EWRL},
  year = {2025},
  url = {https://openreview.net/forum?id=EPt4SnA4pG},
}
Figure