A New Data Source for Inverse Dynamics Learning

Daniel Kappler, Franziska Meier, Nathan D. Ratliff, Stefan Schaal

Abstract

Modern robotics is gravitating toward increasingly collaborative human-robot interaction. Tools such as acceleration policies can naturally support the realization of reactive, adaptive, and compliant robots. These tools require us to model the system dynamics accurately—a difficult task. The fundamental problem remains that simulation and reality diverge; we do not know how to accurately change a robot's state. Thus, recent research on improving inverse dynamics models has been focused on making use of machine learning techniques. Traditional learning techniques train on the actual realized accelerations, instead of the policy's desired accelerations, which is an indirect data source. Here we show how an additional training signal—measured at the desired accelerations—can be derived from a feedback control signal. This effectively creates a second data source for learning inverse dynamics models. Furthermore, we show how both the traditional and this new data source can be used to train task-specific models of the inverse dynamics, when used independently or combined. We analyze the use of both data sources in simulation and demonstrate its effectiveness on a real-world robotic platform. We show that our system incrementally improves the learned inverse dynamics model, and when using both data sources combined, converges more consistently and faster.

Introduction

For a robot to move gracefully and safely around humans, it needs an accurate inverse dynamics model. This model determines the precise torque required at each joint to achieve a specific target acceleration. While traditional physics-based models (Rigid Body Dynamics) provide a foundation, they often fail to account for real-world complexities like friction, cable tension, and varying payloads.

To bridge this gap, researchers employ machine learning to develop "error models." However, traditional learning is indirect: it maps applied torque to the actual achieved acceleration. This creates a catch-22 for compliant robotics: low-gain (soft) controllers are safer but often fail to overcome friction enough to generate the data needed for learning. Conversely, high-gain (stiff) controllers provide better data but are potentially unsafe for human-robot interaction.

Methodology: Joint Learning from Dual Data Sources

This research introduces a Joint Learning approach that leverages two distinct data sources to train a task-specific feedforward neural network:

  1. Indirect Data: The standard approach, mapping the current state $(q, \dot{q})$ and actual acceleration $(\ddot{q}_a)$ to the total applied torque.
  2. Direct Data: The core innovation. By mathematically transforming the feedback control signal—the "correction" applied when the robot deviates from its path—the system derives a training signal at the desired acceleration $(\ddot{q}_d)$.

By combining these sources, the robot learns not just from where it ended up, but from where it was intended to go. This allows the model to be updated iteratively after each task execution, pulling the model toward the target trajectory even when starting with low-gain controllers.

Experimental Evaluation and Results

The approach was evaluated in both a 2D simulation (featuring significant mass discrepancies and friction) and on a real-world 7-DOF KUKA Lightweight Arm.

Average q error over iterations in simulation. The 'joined' approach (blue) consistently outperforms or matches the best of 'indirect' (red) and 'direct' (green) methods across varying noise levels and controller types.

Key Findings

  • Faster Convergence: As shown in the simulation results, the "Joined" approach reached low error levels significantly faster than using either data source in isolation.
  • Robustness to Noise: While direct learning can be sensitive to sensor noise and indirect learning is hindered by friction, the joint approach remained stable across various noise levels.
  • Real-World Performance: On the KUKA arm, the system significantly improved tracking for a heavy-load "pick and place" task after just a single iteration.
  • Enabling Compliance: The method successfully identified dynamics models even with gains an order of magnitude lower than traditional settings, proving that robots do not need to be dangerously stiff to learn effectively.

Discussion and Future Work

The primary insight of this work is that feedback control is more than a corrective mechanism; it is an online measurement of model error. This "feedback as a teacher" paradigm allows for compliance-first learning, where robots can safely learn in human environments.

Future research will address current limitations, such as payload ambiguity, where mid-task changes in weight can confuse the model. Additionally, the team is exploring active learning to dynamically weight the two data sources based on real-time reliability and developing methods for task switching to allow robots to seamlessly transition between different learned dynamics models.

Summary updated

Abstract

Modern robotics is gravitating toward increasingly collaborative human robot interaction. Tools such as acceleration policies can naturally support the realization of reactive, adaptive, and compliant robots. These tools require us to model the system dynamics accurately -- a difficult task. The fundamental problem remains that simulation and reality diverge--we do not know how to accurately change a robot's state. Thus, recent research on improving inverse dynamics models has been focused on making use of machine learning techniques. Traditional learning techniques train on the actual realized accelerations, instead of the policy's desired accelerations, which is an indirect data source. Here we show how an additional training signal -- measured at the desired accelerations -- can be derived from a feedback control signal. This effectively creates a second data source for learning inverse dynamics models. Furthermore, we show how both the traditional and this new data source, can be used to train task-specific models of the inverse dynamics, when used independently or combined. We analyze the use of both data sources in simulation and demonstrate its effectiveness on a real-world robotic platform. We show that our system incrementally improves the learned inverse dynamics model, and when using both data sources combined converges more consistently and faster.
BibTeX
@inproceedings{kappler2017new,
  author = {Daniel Kappler and Franziska Meier and Nathan D. Ratliff and Stefan Schaal},
  title = {{A New Data Source for Inverse Dynamics Learning}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2017},
  url = {https://openreview.net/forum?id=h8IKE98DXj},
}
Figure