Bimanual Rope Manipulation Skill Synthesis through Context Dependent Correction Policy Learning from Human Demonstration

T. Baturhan Akbulut, Gülsüm Tuba Çibuk Girgin, Arash Mehrabi, Minoru Asada, Emre Ugur, Erhan Öztop

Introduction

Learning from Demonstration (LfD) has emerged as a powerful paradigm for teaching robots complex skills. However, long-horizon tasks such as bimanual rope manipulation frequently suffer from "compounding errors." A minor deviation early in a sequence can lead to a state significantly different from the training distribution—a phenomenon known as covariate shift—resulting in total task failure. While large-scale data collection or reinforcement learning are common remedies, they are often data-inefficient. This work proposes a more targeted approach: identifying and learning specialized "correction policies" for critical task transitions.

Proposed Framework: Strategic Segmentation

The authors argue that complex tasks are best modeled as a sequence of primitives where the transitions between them are the most vulnerable points. They introduce a framework that segments a task into three distinct phases:

  1. Pre-Movement Primitive (pre-MP): The initial trajectory toward a goal.
  2. Corrective Motor Primitive (cor-MP): A context-dependent action triggered when the robot fails to meet the preconditions of the next step.
  3. Post-Movement Primitive (post-MP): The final action sequence to complete the task.
Conceptual diagram of the task segmentation into Pre-MP, Cor-MP (corrective), and Post-MP trajectories.

Central to this framework is the Conditional Neural Motor Primitive (CNMP). CNMPs are deep learning architectures capable of representing sensorimotor data and generating trajectories conditioned on environmental "context," such as object positions or camera images processed via an Autoencoder. By focusing human demonstrations on these "corrections" at critical points, the robot learns to recover from errors without requiring exhaustive demonstrations of the entire task.

Experimental Evaluation

The methodology was validated across two distinct environments:

Simulation: Object Manipulation

In a Gazebo simulation, a Torobo humanoid robot was tasked with pushing an object through a corridor. An Autoencoder (AE) was trained on 400 images from the robot's head camera to extract latent context vectors. The proposed method achieved a 100% success rate, significantly outperforming a monolithic baseline that attempted to learn the entire task as a single policy (44-90% success).

Real-World: Bimanual Rope Knotting

The system was further tested on a complex bimanual rope-knotting task. Context was defined by the relative position of the rope end to the gripper, extracted via color segmentation.

A Torobo humanoid robot performing a bimanual rope-knotting task over a horizontal bar.

The robot successfully performed the knotting task even when the rope was placed in novel positions not seen during the 28 corrective demonstrations. This demonstrates the system's ability to interpolate between known corrections to handle unseen errors.

Key Insights and Limitations

The research highlights that modularity is more efficient than monolithic policy learning. By focusing on critical transitions, the robot becomes resilient with minimal data. However, several challenges remain. The "critical points" for segmentation are currently identified manually by humans. Additionally, real-world execution is sometimes hindered by kinematic constraints (joint limits) or visual occlusions during complex bimanual maneuvers. Future work aims to automate the detection of these critical regions and improve visual robustness.

Summary updated

Abstract

Learning from demonstration (LfD) provides a convenient means to equip robots with dexterous skills when demonstration can be obtained in robot intrinsic coordinates. However, the problem of compounding errors in long and complex skills reduces its wide deployment. Since most such complex skills are composed of smaller movements that are combined, considering the target skill as a sequence of compact motor primitives seems reasonable. Here the problem that needs to be tackled is to ensure that a motor primitive ends in a state that allows the successful execution of the subsequent primitive. In this study, we focus on this problem by proposing to learn an explicit correction policy when the expected transition state between primitives is not achieved. The correction policy is itself learned via behavior cloning by the use of a state-of-the-art movement primitive learning architecture, Conditional Neural Motor Primitives (CNMPs). The learned correction policy is then able to produce diverse movement trajectories in a context dependent way. The advantage of the proposed system over learning the complete task as a single action is shown with a table-top setup in simulation, where an object has to be pushed through a corridor in two steps. Then, the applicability of the proposed method to bi-manual knotting in the real world is shown by equipping an upper-body humanoid robot with the skill of making knots over a bar in 3D space. The experiments show that the robot can perform successful knotting even when the faced correction cases are not part of the human demonstration set.
BibTeX
@inproceedings{akbulut2022bimanual,
  author = {T. Baturhan Akbulut and Gülsüm Tuba Çibuk Girgin and Arash Mehrabi and Minoru Asada and Emre Ugur and Erhan Öztop},
  title = {{Bimanual Rope Manipulation Skill Synthesis through Context Dependent Correction Policy Learning from Human Demonstration}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2022},
}
Figure