Towards Coordinated Robot Motions: End-to-End Learning of Motion Policies on Transform Trees

Muhammad Asif Rana, Anqi Li, Dieter Fox, Sonia Chernova, Byron Boots, Nathan D. Ratliff

Abstract

This paper introduces a framework for learning complex, multi-task robot motions from human demonstrations by combining structured Riemannian Motion Policies (RMPs) with end-to-end deep learning. By optimizing subtask policies jointly rather than independently, the system learns to balance competing objectives—such as reaching a goal while maintaining a specific orientation—while providing mathematical guarantees for motion stability.

Introduction

Teaching robots to perform complex maneuvers, such as picking an object from a cramped shelf and placing it precisely elsewhere, is challenging because these tasks involve multiple simultaneous constraints. A robot must reach a target, avoid collisions, and manage its internal joint limits simultaneously. While Learning from Demonstration (LfD) is a popular approach, standard methods often struggle with inconsistent demonstrations in joint space and the difficulty of coordinating modular subtasks.

Methodology

The proposed framework builds upon RMPflow, treating robot motion as a flow on a manifold. The key innovation lies in the use of Transform Trees and end-to-end learning to coordinate subtasks.

Transform Trees

The system uses a tree structure to map the robot's configuration space to various subtask spaces. This allows users to specify exactly which parts of the robot (e.g., gripper tips, elbow) are relevant for a specific task.

A transform tree mapping the robot's configuration space to various subtask manifolds.

End-to-End Learning

Unlike previous work that learned subtask policies independently, this method uses a joint objective function. It differentiates through the entire motion generation process to learn the importance weights (metrics) that coordinate different subtasks. This hybrid approach allows for the mixing of learned policies with hand-designed constraints, providing both flexibility and safety. To ensure the learned policies are expressive yet stable, the authors employ:

  • Diffeomorphism Networks: Mapping complex motions to a simple latent space.
  • Metric Networks: Ensuring the importance of each task is mathematically well-behaved (positive definite).
Neural network architectures for the diffeomorphism and metric components.

Experimental Results

The approach was validated using a 7-DOF Rethink Sawyer robot and human demonstrations collected via kinesthetic teaching. The tasks included constrained inspection and multi-shelf placing.

The "Coordinated" approach significantly outperformed baselines in both accuracy and generalization. In complex placing tasks, it reduced orientation error by nearly half compared to independent learning. When starting from novel positions, the coordinated approach achieved a near 100% success rate, whereas baseline methods often dropped to 10-20%. Visualizations confirm that the model produces smooth, human-like trajectories that successfully reach the goal without getting "stuck."

Conclusion

By integrating geometric structure with deep learning, this framework enables robots to learn sophisticated coordination from human data. The use of natural gradient flow and diffeomorphisms ensures that the system is Lyapunov stable, meaning the robot will always converge to its goal without requiring extensive hand-tuning of safety parameters. Future work will focus on dynamic environments and automated structure discovery.

Summary updated

Abstract

Generating robot motion that fulfills multiple tasks simultaneously is challenging due to the geometric constraints imposed by the robot. In this paper, we propose to solve multi-task problems through learning structured policies from human demonstrations. Our structured policy is inspired by RMPflow, a framework for combining subtask policies on different spaces. The policy structure provides the user an interface to 1) specifying the spaces that are directly relevant to the completion of the tasks, and 2) designing policies for certain tasks that do not need to be learned. We derive an end-to-end learning objective function that is suitable for the multi-task problem, emphasizing the deviation of motions on task spaces. Furthermore, the motion generated from the learned policy class is guaranteed to be stable. We validate the effectiveness of our proposed learning framework through qualitative and quantitative evaluations on three robotic tasks on a 7-DOF Rethink Sawyer robot.
BibTeX
@inproceedings{rana2021towards,
  author = {Muhammad Asif Rana and Anqi Li and Dieter Fox and Sonia Chernova and Byron Boots and Nathan D. Ratliff},
  title = {{Towards Coordinated Robot Motions: End-to-End Learning of Motion Policies on Transform Trees}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=K0l2HRumhs},
}
Figure