TL;DR
CADENCE is a learned step planner that allows quadruped robots to navigate challenging, hole-filled terrain by abandoning rigid, cyclic gait schedules in favor of flexible, acyclic stepping. By pairing reinforcement learning with a novel "Inverse Footstep Model," the system can be trained on one controller and successfully transferred to a completely different one with minimal additional data.
What Problem Does This Paper Address?
Most quadruped robots rely on model-based controllers that follow a fixed, cyclic gait—like a trot or a walk. While these controllers are robust, they are inherently rigid: they lift legs in a specific order on a fixed timer. On "sparse" terrain (ground riddled with holes or "voids"), this rigidity is a liability. A cyclic gait might command a robot to step exactly where there is no ground, leading to a fall.
Ideally, a robot should decide when and where to step based on the terrain in front of it. However, making these decisions online is computationally expensive for traditional optimization-based controllers. Furthermore, a planning policy trained to work with one specific controller often fails when moved to another because different controllers have different "landing errors"—biases in where the foot actually touches down versus where it was commanded to go.
Methods
The authors introduce CADENCE, a learned step planner that sits on top of existing model-based controllers.
1. A Small, Discrete Action Space
Instead of complex joint trajectories, CADENCE makes a simple choice every 40 ms: it either waits or starts one step. It chooses from at most 65 discrete actions: "no step" or one of 16 "safe" candidate footholds for each of the four legs. This small action space allows for efficient Reinforcement Learning (RL) using Proximal Policy Optimization (PPO).
2. The Candidate Mixer
To decide which action is best, CADENCE uses a candidate mixer—a small transformer encoder. This architecture allows the policy to "attend" to all possible choices simultaneously, comparing a potential step for the front-left leg against the option of simply waiting or stepping with the back-right leg.
3. Inverse Footstep Model (IFM)
The "secret sauce" for transferability is the IFM. It's a small neural network that learns the specific landing biases of a controller. When CADENCE picks a target foothold, the IFM adjusts the command sent to the controller to ensure the foot actually lands on target. This allows a "frozen" CADENCE policy to move from a simple convex MPC (used for fast training) to a complex nonlinear MPC (like OCS2) just by refitting the IFM on a small amount of data (~20,000 steps).
Datasets
The system was trained and evaluated in simulation:
- Training: Conducted in Isaac Lab on a curriculum of terrains with increasing "void fractions" (up to 40% of the ground missing).
- Evaluation: A benchmark was created in Gazebo using the OCS2 control stack. Terrains featured void fractions of 26%, 38%, and 49%, with the robot moving at speeds of 0.1 m/s and 0.2 m/s.
Key Insights and Ideas
- Emergent Acyclicity: Even though the reward function had no "gait term" (it only penalized steps and failures), the robot naturally developed an acyclic gait. It didn't follow a fixed period or order, yet it still "discovered" that pairing diagonal legs—similar to a trot—is an efficient way to move.
- The Power of Waiting: By having a "no-step" option every 40ms, the robot can pause a leg's swing if no safe foothold is reachable, or speed up its stepping frequency to cross a dense patch of holes.
- Decoupling Planning from Control: The IFM effectively acts as a translator between the high-level "where I want to go" and the low-level "how this specific controller moves." This decoupling is what makes the policy portable across different robotic software stacks.
Key Results
CADENCE significantly outperformed standard cyclic gaits on the OCS2 benchmark:
- Overall Success: CADENCE achieved an 85.1% success rate, compared to 67.8% for a standard trot and 65.5% for a static walk.
- Extreme Terrain: In the most difficult scenario (49% voids), CADENCE maintained a 72.9% success rate, while the standard trot dropped to 43.0%.
- Landing Accuracy: On flat ground, the refitted IFM reduced the RMS landing error from 6 cm to just 2 cm, which is critical when stepping on small "islands" of safe ground.
Limitations and Future Work
- Flat Ground Only: The current work focuses on level ground with holes. The authors plan to extend the candidate sampling to handle stairs and slopes in the future.
- Data-Dependent Transfer: While refitting the IFM is much faster than retraining the whole policy, it still requires logging about 20,000 steps on the new controller. Future work may explore "data-free" transfer or online adaptation.
- Sim-to-Real: The paper evaluates the system in high-fidelity simulations (Gazebo/OCS2), but physical hardware tests remain for future validation.
Summary updated