Metric-Based Imitation Learning Between Two Dissimilar Anthropomorphic Robotic Arms

Marcus Ebner von Eschenbach, Binyamin Manela, Jan Peters, Armin BiessarXiv 2020 · arXiv

Introduction

Imitation learning typically assumes a direct mapping between an expert's actions and a learner's capabilities. However, when the expert and learner possess different physical structures—such as different joint counts or limb lengths—the "correspondence problem" arises. Traditional solutions like kinesthetic teaching or manual teleoperation are resource-intensive and lack scalability. This paper proposes a mathematical framework to automate similarity mapping between dissimilar robotic embodiments, allowing a learner to translate an expert's spatial configuration into its own unique state space.

Methodology

The authors introduce a metric-based framework that avoids manual joint-to-joint mapping by focusing on global spatial similarity.

Embodiment State and Distance Metric

The state of a robot is represented by frames (homogeneous transformation matrices in $SE(3)$) and twists (velocities in $se(3)$) for each link. A composite distance function $d(s, \hat{s})$ quantifies the difference between the expert state $s$ and learner state $\hat{s}$ using three components: translational distance, rotational distance (using a cosine-based measure for stability), and velocity distance.

State representation using frames and twists on planar manipulators.

To handle varying link counts, a correspondence matrix $W$ is employed. This can be a static pre-calculated mapping or a "soft" state-dependent mapping using a softmax function to identify nearest-neighbor links in real-time. This "softmax trick" ensures smooth transitions and prevents jerky movements during imitation.

Learning Architectures

The framework supports two learning modes:

  • Static Pose Imitation: A neural network learns to map expert joint angles to learner joint angles by minimizing the distance metric as a loss function.
  • Dynamic Movement Imitation: The distance metric serves as a negative reward signal within a Proximal Policy Optimization (PPO) reinforcement learning loop, enabling the agent to learn trajectories without prior knowledge of its own dynamics.

Experiments and Results

The framework was validated using 2D planar manipulators and high-fidelity simulations of Franka Emika Panda arms in Gazebo.

Imitation of a 7-DOF expert by a learner with restricted degrees of freedom.

Pose and Trajectory Generalization

Neural networks successfully generalized pose translations from a 7-DOF expert to a 4-DOF learner. In dynamic tasks, the PPO agent learned to follow complex trajectories. Notably, training on a diverse set of trajectories significantly reduced generalization error compared to single-trajectory training. Even when restricted to only 3 DOFs, the learner successfully minimized the distance metric by aligning its available links with the general spatial "silhouette" of the 7-DOF expert.

Conclusion

This research demonstrates that a well-defined spatial distance metric can bridge the gap between dissimilar robotic bodies. While the optimization landscape can occasionally suffer from local minima, the combination of rotational alignment and static correspondence provides a robust foundation for automated imitation learning. Future work may extend this "free topology" approach to more diverse robotic forms, such as swarms or non-anthropomorphic systems.

Summary updated

Abstract

The development of autonomous robotic systems that can learn from human demonstrations to imitate a desired behavior - rather than being manually programmed - has huge technological potential. One major challenge in imitation learning is the correspondence problem: how to establish corresponding states and actions between expert and learner, when the embodiments of the agents are different (morphology, dynamics, degrees of freedom, etc.). Many existing approaches in imitation learning circumvent the correspondence problem, for example, kinesthetic teaching or teleoperation, which are performed on the robot. In this work we explicitly address the correspondence problem by introducing a distance measure between dissimilar embodiments. This measure is then used as a loss function for static pose imitation and as a feedback signal within a model-free deep reinforcement learning framework for dynamic movement imitation between two anthropomorphic robotic arms in simulation. We find that the measure is well suited for describing the similarity between embodiments and for learning imitation policies by distance minimization.
BibTeX
@misc{eschenbach2020metric,
  author = {Marcus Ebner von Eschenbach and Binyamin Manela and Jan Peters and Armin Biess},
  title = {{Metric-Based Imitation Learning Between Two Dissimilar Anthropomorphic Robotic Arms}},
  year = {2020},
  eprint = {2003.02638},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2003.02638},
}
Figure