Data-Driven Adaptation for Robust Bipedal Locomotion with Step-to-Step Dynamics

Min Dai, Xiaobin Xiong, Jaemin Lee, Aaron D. Ames

TL;DR

This paper introduces an online, data-driven framework for bipedal robots to adapt their walking gait in real-time to unknown environments and model uncertainties. By learning step-to-step dynamics online, the system achieves more robust and agile locomotion compared to traditional methods, even under significant disturbances like added weight or external forces.

What Problem Does This Paper Address?

Bipedal robots are notoriously difficult to control, especially when walking in unpredictable environments. Traditional control methods often rely on precise models of the robot and its surroundings, which are rarely available in real-world scenarios. While model-free approaches like reinforcement learning can learn robust gaits, they typically require massive amounts of data and are sensitive to reward design. Existing data-driven methods often require offline data collection, limiting their ability to adapt to unforeseen changes during operation. This paper tackles the challenge of creating a control system that can learn and adapt its walking strategy online to handle uncertainties and disturbances without prior knowledge of their nature or magnitude.

Methods

The core of the proposed method lies in an adaptive control approach applied to the step-to-step (S2S) dynamics of bipedal locomotion. S2S dynamics captures how a robot's state (e.g., center of mass position and velocity) evolves from one foot-strike to the next.

Here's a breakdown of the key techniques:

  • Step-to-Step (S2S) Dynamics: Instead of controlling the full, complex continuous dynamics of the robot, the system focuses on the discrete transitions between steps. This simplifies the problem.
  • Hybrid Linear Inverted Pendulum (H-LIP) Model: A simplified, linear model used as a baseline. It's effective for basic walking but struggles with uncertainties.
  • Data-Driven S2S Model Learning: The paper proposes learning a robot-specific S2S model from data. Crucially, this learning happens online.
  • Adaptive Control: The system uses classical adaptive control techniques to iteratively update its estimate of the robot's S2S dynamics. This allows it to adjust its control strategy as it gathers new data during walking.
  • Two Formulations:
    • State Tracking: The adaptive controller directly learns the robot's S2S dynamics and uses this to generate stable step plans.
  • Output Tracking: This formulation accounts for errors introduced by the low-level controller that executes the planned steps. It aims to ensure the actual foot placement matches the desired trajectory, leading to better overall velocity tracking.
  • Output Construction: Once the S2S dynamics are learned and a step plan is generated, the system constructs continuous swing foot trajectories and other desired motions using Bézier polynomials. These are then tracked by a low-level controller (e.g., Quadratic Program).

What's new is the online adaptation of the S2S dynamics model, allowing the robot to continuously refine its walking strategy based on real-time experience, rather than relying on pre-collected data or a fixed model.

Datasets

The paper primarily uses simulation data generated from a high-fidelity model of the Cassie bipedal robot. They do not introduce new datasets but rather leverage the simulation environment to test their approach under various controlled conditions.

Key Insights and Ideas

  • Online Adaptation is Key for Robustness: The most significant insight is that learning the robot's dynamics while it's walking is crucial for handling unexpected changes. A fixed, even if data-driven, model cannot cope with novel disturbances.
  • S2S Dynamics as a Powerful Abstraction: Focusing on step-to-step dynamics simplifies the complex problem of bipedal locomotion, making it amenable to adaptive control techniques.
  • Output Tracking Improves Performance: The "output tracking" formulation, which considers the actual executed step, provides better velocity tracking than simply trying to match the planned step sizes, especially when the low-level execution is imperfect.
  • Disturbance Agnosticism: The adaptive approach doesn't need to know what the disturbance is (e.g., extra weight, slope, drag force) or its magnitude. It simply learns how the dynamics change and adjusts accordingly.

Key Results

The paper demonstrates significant improvements in robust locomotion across various challenging scenarios:

  • Velocity Tracking: The adaptive controller significantly improves velocity tracking accuracy compared to a baseline H-LIP controller, especially under varying gait parameters.
  • Heavy Loads: The robot can maintain stable walking with an additional 10kg load (30% of its weight) without any explicit knowledge of the load.
  • Modified Mass-Inertia: The system successfully adapts to changes in leg mass and inertia, with the output tracking formulation showing superior performance.
  • State Estimation Error: The adaptive controller effectively compensates for a constant bias in the estimated forward velocity, achieving much lower tracking errors than the baseline.
  • External Drag Forces: The robot can handle significant sagittal (up to 100N) and lateral (up to 50N) drag forces, maintaining stable stepping-in-place motion, whereas the baseline controller fails.
  • Sloped Terrains: The adaptive controller successfully adapts to walking on slopes of ±0.1 rad, a task that challenges the nominal H-LIP controller.

Limitations and Future Work

  • Low-Level Controller Sensitivity: The output tracking formulation, while effective, can be sensitive to the choice of adaptive gain and the speed of reference velocity changes, potentially leading to oscillations. The authors hypothesize that the linear model for tracking error might be too simplistic.
  • Persistent Disturbances: The current evaluation focuses on persistent disturbances. The paper suggests future work could explore impulse-type disturbances, which can be rejected by the proposed state feedback control within the robot's kinematic and dynamic feasibility.
  • Integration of Tracking Error: The authors plan to explore alternative methods to better integrate the tracking error from the low-level controller into their adaptive framework.
  • Experimental Validation: While simulations are extensive, the paper mentions plans to report experimental results shortly.

Summary updated

Abstract

This paper presents an online framework for synthesizing agile locomotion for bipedal robots that adapts to unknown environments, modeling errors, and external disturbances. To this end, we leverage step-to-step (S2S) dynamics which has proven effective in realizing dynamic walking on underactuated robots -- assuming known dynamics and environments. This paper considers the case of uncertain models and environments and presents a data-driven representation of the S2S dynamics that can be learned via an adaptive control approach that is both data-efficient and easy to implement. The learned S2S controller generates desired discrete foot placement, which is then realized on the full-order dynamics of the bipedal robot by tracking desired outputs synthesized from the given foot placement. The benefits of the proposed approach are twofold. First, it improves the ability of the robot to walk at a given desired velocity when compared to the non-adaptive baseline controller. Second, the data-driven approach enables stable and agile locomotion under the effect of various unknown disturbances: additional unmodeled payload, large robot model errors, external disturbance forces, biased velocity estimation, and sloped terrains. This is demonstrated through in-depth evaluation with a high-fidelity simulation of the bipedal robot Cassie subject to the aforementioned disturbances.
BibTeX
@inproceedings{dai2023data,
  author = {Min Dai and Xiaobin Xiong and Jaemin Lee and Aaron D. Ames},
  title = {{Data-Driven Adaptation for Robust Bipedal Locomotion with Step-to-Step Dynamics}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2023},
  doi = {10.1109/iros55552.2023.10341396},
}
Figure