Biped Walk Learning Through Playback and Corrective Demonstration

Çetin Meriçli, Manuela M. Veloso

Abstract

Bipedal locomotion remains a significant challenge in robotics due to the discrepancies between idealized analytical models and real-world physical dynamics. This paper proposes a pragmatic "human-in-the-loop" approach to stabilize the Aldebaran Nao robot's walk. The method involves two phases: recording a successful open-loop walk cycle and subsequently learning a closed-loop stabilization policy through real-time human corrective feedback provided via a Nintendo Wiimote. By mapping human intuition to onboard accelerometer data, the robot achieves a walking distance of 956 cm, significantly outperforming both open-loop playback (327 cm) and expert hand-tuned controllers (689 cm).

Introduction

Maintaining stability in bipedal robots is a high-dimensional control problem. While analytical models provide a mathematical foundation for movement, they often fail in practice due to unmodeled factors such as gear friction, sensor noise, and the complex dynamics of the robot's own mass distribution. The Aldebaran Nao robot, used in this study, features a heavy torso and lacks certain degrees of freedom (like hip yaw joints), making manual controller tuning particularly arduous.

The Aldebaran Nao robot (a) and its associated coordinate system (b) used for stabilization mapping.

The Modeling Gap

The primary obstacle to stable walking is the "reality gap" between commanded joint angles and their actual execution. As shown in Figure 4, there is a distinct lag and deviation between the intended joint command and the actual position reached by the robot's hardware. These small errors accumulate over multiple walk cycles, eventually leading to a loss of balance and a fall.

Comparison of joint commands versus actual positions for the left ankle roll, highlighting the discrepancies caused by real-world physical constraints.

Methodology

The researchers implemented a two-phase learning framework to bridge this gap:

Phase One: Open-Loop Playback

The process begins by executing a standard analytical walk algorithm. The system records a single, successful "walk cycle" (a sequence of joint commands) from trials where the robot remained upright. This cycle is then played back in an infinite loop. While this provides a baseline movement, it lacks the feedback necessary to recover from perturbations.

Phase Two: Corrective Learning

To introduce closed-loop stability, a human teacher provides real-time corrections.

  • Interface: Using a Nintendo Wiimote and Nunchuk, the teacher observes the robot and provides tilt corrections (forward/backward and side-to-side). This wireless interface ensures the teacher does not physically interfere with the robot's dynamics.
  • Data Mapping: The robot records these human-initiated corrections alongside its internal accelerometer data at 50Hz.
  • Policy Generation: The system fits a normal distribution to the correction data for each sensor state. During autonomous operation, the robot applies the mean correction corresponding to its current accelerometer reading to its joint commands.

To ensure smooth movement, a running mean filter (window size of 5) is applied to the sensor data, and a geometric constraint is used to couple hip and ankle movements ($C_{ankle} = -C_{hip}$), keeping the feet flat during torso stabilization.

Experimental Results

The effectiveness of the learned policy was evaluated over 20 trials, comparing it against open-loop playback and a manually tuned closed-loop controller.

Method Maximum Distance Traveled (cm)
Open-loop Playback 327
Hand-tuned Closed-loop 689
Learned Closed-loop (Proposed) 956

The results indicate that the learned policy not only provides superior stability compared to the baseline but also outperforms expert-level manual tuning. This suggests that human intuition can effectively capture and compensate for complex physical nuances that are difficult to model analytically.

Conclusion

This research demonstrates that human corrective demonstration is a powerful tool for robot learning in high-dimensional spaces. By leveraging a simple wireless interface and mapping human reactions to onboard sensors, the Nao robot achieved a significantly more robust walk. Future work will explore extending this framework to multi-directional walking and more complex "Advice Operators" for multi-joint coordination.

Summary updated

Abstract

Developing a robust, flexible, closed-loop walking algorithm for a humanoid robot is a challenging task due to the complex dynamics of the general biped walk. Common analytical approaches to biped walk use simplified models of the physical reality. Such approaches are partially successful as they lead to failures of the robot walk in terms of unavoidable falls. Instead of further refining the analytical models, in this work we investigate the use of human corrective demonstrations, as we realize that a human can visually detect when the robot may be falling. We contribute a two-phase biped walk learning approach, which we experiment on the Aldebaran NAO humanoid robot. In the first phase, the robot walks following an analytical simplified walk algorithm, which is used as a black box, and we identify and save a walk cycle as joint motion commands. We then show how the robot can repeatedly and successfully play back the recorded motion cycle, even if in open-loop. In the second phase, we create a closed-loop walk by modifying the recorded walk cycle to respond to sensory data. The algorithm learns joint movement corrections to the open-loop walk based on the corrective feedback provided by a human, and on the sensory data, while walking autonomously. In our experimental results, we show that the learned closed-loop walking policy outperforms a hand-tuned closed-loop policy and the open-loop playback walk, in terms of the distance traveled by the robot without falling.
BibTeX
@inproceedings{mericli2010biped,
  author = {Çetin Meriçli and Manuela M. Veloso},
  title = {{Biped Walk Learning Through Playback and Corrective Demonstration}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2010},
  doi = {10.1609/aaai.v24i1.7730},
}
Figure