Data-driven Step-to-step Dynamics based Adaptive Control for Robust and Versatile Underactuated Bipedal Robotic Walking

Min Dai, Xiaobin Xiong, Aaron D. Ames

Abstract

This paper introduces an online adaptive framework that enables bipedal robots to walk robustly by learning their own "step-to-step" (S2S) dynamics on the fly. By combining the simplicity of reduced-order physics models with real-time parameter estimation, the authors enable the robot Cassie to maintain stable locomotion despite heavy payloads, sloped terrain, and significant external push forces—all without needing massive reinforcement learning datasets.

Introduction

Bipedal walking is a complex control problem due to the "underactuated" nature of the systems and the "hybrid" physics involved in switching between continuous motion and discrete foot impacts. Traditional model-based approaches often rely on simplified models like the Linear Inverted Pendulum (LIP). While computationally efficient, these models are often too generic to account for specific robot characteristics such as leg mass, motor friction, or environmental changes like carrying a heavy payload.

Conversely, data-driven methods like Reinforcement Learning (RL) offer robustness but typically require millions of simulation steps and function as "black boxes" that are difficult to interpret or tune. This research bridges the gap by proposing a method that utilizes a simple linear model while updating its parameters at every step.

Cassie robot navigating various challenging environments including unknown loads, slopes, and external forces.

Methodology: Step-to-Step (S2S) Dynamics

The core of the proposed approach is the Step-to-Step (S2S) Dynamics framework. Rather than attempting high-frequency control at every millisecond, the framework focuses on the robot's state at the discrete moment of foot-strike.

The Model and Adaptation

The robot's motion is represented as a linear system: $x_k = A x_{k-1} + B u_{k-1} + C$ where $x$ represents the center-of-mass state and $u$ is the step size. To handle real-time changes, a Projection Algorithm (PA)—a classical adaptive control technique—is employed to update the matrices $A, B,$ and $C$ at every step based on the discrepancy between predicted and actual foot placement.

Formulations

The framework explores two primary formulations:

  1. State Tracking: Focuses on learning the underlying physical dynamics of the robot.
  2. Output Tracking: An advanced version that accounts for tracking errors from low-level motor controllers, such as when a leg does not land exactly as commanded.

The high-level adaptive controller determines the optimal step placement, which is then translated into specific motor torques by a low-level Quadratic Program (QP) controller operating at 1kHz.

Experimental Evaluation

The framework was validated using a high-fidelity simulation of the Cassie bipedal robot (Agility Robotics) within the MuJoCo physics engine. The "data" for adaptation is generated live during simulations across several challenging scenarios:

  • 10kg payloads (approx. 30% of body weight).
  • 2.5x increased leg mass.
  • 100N frontal drag forces and 50N lateral forces.
  • Slopes of $\pm 0.1$ rad (approx. 5.7 degrees).

Key Results and Insights

The adaptive controller demonstrated significant improvements over non-adaptive baselines:

  • Payload and Mass Resilience: Cassie successfully maintained balance with a 10kg load and increased leg mass, conditions under which the baseline H-LIP controller failed.
  • Disturbance Rejection: The system maintained stability under constant 100N sagittal and 50N lateral forces.
  • Environmental Adaptation: The robot navigated both up-slopes and down-slopes by correcting for timing variations in foot impacts.
  • Sensor Bias Handling: The model proved capable of "absorbing" sensor errors. For instance, if velocity estimates were off by 0.4 m/s, the adaptive parameters adjusted to allow the robot to maintain the correct physical speed.

A critical insight was the advantage of Output Tracking. By incorporating the imperfections of the low-level controller into the adaptive model, the robot achieved much higher precision in foot placement compared to models that only learned the physics.

Conclusion and Future Work

The proposed adaptive S2S framework offers a data-efficient alternative to RL, refining its control strategy within seconds of walking. Its linear formulation ensures it is computationally light enough for real-time onboard execution.

However, limitations remain. The current framework is optimized for persistent disturbances rather than sudden impulse forces (like a quick shove). Additionally, the assumption of linear S2S dynamics may not hold for highly non-linear maneuvers such as jumping or high-speed running. Future work will focus on hardware implementation and extending the model to more dynamic behaviors.

Summary updated

BibTeX
@inproceedings{dai2023data,
  author = {Min Dai and Xiaobin Xiong and Aaron D. Ames},
  title = {{Data-driven Step-to-step Dynamics based Adaptive Control for Robust and Versatile Underactuated Bipedal Robotic Walking}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2023},
  doi = {10.1109/iros55552.2023.10341396},
}
Figure