Probabilistic Model Learning and Long-term Prediction for Contact-rich Manipulation Tasks

Shahbaz Abdul Khader, Hang Yin, Pietro Falco, Danica Kragic

Abstract

In this letter, we investigate learning forward dynamics models and multi-step prediction of state variables (long-term prediction) for contact-rich manipulation. The problems are formulated in the context of model-based reinforcement learning (MBRL). We focus on two aspects—discontinuous dynamics and data-efficiency—both of which are important in the identified scope and pose significant challenges to State-of-the-Art methods. We contribute to closing this gap by proposing a method that explicitly adopts a specific hybrid structure for the model while leveraging the uncertainty representation and data-efficiency of Gaussian process. Our experiments on an illustrative moving block task and a 7-DOF robot demonstrate a clear advantage when compared to popular baselines in low data regimes.

The Challenge of Contact-Rich Manipulation

In robotics, "contact-rich" tasks—where a robot frequently touches, slides against, or impacts objects—are notoriously difficult to model. These interactions create discontinuous dynamics: a sudden impact causes an abrupt jump in velocity that standard smooth mathematical models cannot easily capture.

A robot arm performing a contact-rich task by pushing a block.

Furthermore, most modern approaches, such as those used in Model-Based Reinforcement Learning (MBRL), require massive amounts of data. In real-world scenarios, running thousands of trials to learn a model is impractical due to physical wear and tear on the robot. The challenge lies in creating a model that is both data-efficient and capable of long-term prediction (forecasting many steps into the future) despite these sharp "jumps" in system behavior.

Methodology: A Hybrid Probabilistic Framework

The proposed method treats the robot's environment as a Hybrid Dynamical System. Instead of one monolithic model, the task is decomposed into discrete "modes" (e.g., free motion, sliding, or stuck).

  1. Mode Discovery: A Dirichlet Process Gaussian Mixture Model (DPGMM) is used to automatically cluster trial data into different physical modes without human labeling.
  2. Mode Dynamics: Each mode is modeled using a Gaussian Process (GP). GPs are chosen for their extreme data efficiency and ability to provide a measure of uncertainty.
  3. Guard Functions: A Multi-class Support Vector Machine (SVM) acts as a "guard," predicting when the system is likely to switch from one mode to another based on the current state and action.
  4. Reset Maps: This is a critical addition. When a transition occurs (like hitting a wall), a specific GP "reset map" calculates the discontinuous jump in state, such as a sudden loss of velocity.
  5. Uncertainty Propagation: To predict far into the future, the framework uses the Unscented Transform (UT) and Monte Carlo sampling to pass probability distributions through these non-linear models.
Graphical model of state transitions and a plot illustrating discontinuous state jumps with uncertainty.

Experimental Evaluation

The researchers evaluated their approach on two main setups:

  • Simulated 1D Sliding Mass: A block subjected to frictionless motion, sticking, and slipping, serving as a controlled environment to test handling of abrupt velocity jumps.
  • 7-DOF YuMi Robot Experiment: Real-world data from an ABB YuMi robot performing a peg-in-hole style task involving free motion and sliding along a steel edge.

To specifically measure data efficiency, the models were tested in both "small" (15 trials) and "large" (40 trials) data regimes.

Key Results and Insights

  • The Power of Reset Maps: Most learning models try to "smooth over" discontinuities. By explicitly using reset maps, this model acknowledges that impacts are instantaneous jumps, leading to much more realistic simulations of contact.
  • Probabilistic Switching: Because the exact moment of contact is uncertain, the model allows for "multimodal" predictions, tracking multiple possible outcomes (e.g., contact vs. free motion) simultaneously.
  • Superior Accuracy and Efficiency: In the YuMi robot task, the proposed method achieved an RMSE of 0.15, significantly outperforming uncertainty-aware neural networks (uANN) which scored 0.47. Furthermore, the hybrid model performed better with just 15 trials than the baselines did with 40, proving its utility for rapid real-world deployment.

Conclusion and Future Work

By imposing a hybrid automata structure as a "strong prior," the authors enable robots to learn complex contact physics from a handful of examples. While highly effective in low-data regimes, future work will need to address the cubic scaling of Gaussian Processes for larger datasets and the challenge of adapting to previously unseen physical modes.

Summary updated

Abstract

In this letter, we investigate learning forward dynamics models and multi-step prediction of state variables (long-term prediction) for contact-rich manipulation. The problems are formulated in the context of model-based reinforcement learning (MBRL). We focus on two aspects-discontinuous dynamics and data-efficiency-both of which are important in the identified scope and pose significant challenges to State-of-the-Art methods. We contribute to closing this gap by proposing a method that explicitly adopts a specific hybrid structure for the model while leveraging the uncertainty representation and data-efficiency of Gaussian process. Our experiments on an illustrative moving block task and a 7-DOF robot demonstrate a clear advantage when compared to popular baselines in low data regimes.
BibTeX
@article{khader2020probabilistic,
  author = {Shahbaz Abdul Khader and Hang Yin and Pietro Falco and Danica Kragic},
  title = {{Probabilistic Model Learning and Long-term Prediction for Contact-rich Manipulation Tasks}},
  journal = {IEEE Robotics and Automation Letters},
  year = {2020},
  url = {https://openreview.net/forum?id=pfbaioAGBS},
}
Figure