Combining Learning from Demonstration with Learning by Exploration to Facilitate Contact-Rich Tasks

Yunlei Shi, Zhaopeng Chen, Yansong Wu, Dimitri Henkel, Sebastian Riedel, Hongxu Liu, Qian Feng, Jianwei Zhang

Introduction

In modern industrial automation, programming robots for contact-rich tasks—such as inserting electronic components or mechanical parts into tight-fitting sockets—remains a significant hurdle. These tasks require a high degree of precision and the ability to handle physical interactions safely. Traditional methods like teach-pendant programming or hand-guiding often struggle in constrained environments or fail to account for the inherent positioning uncertainties of mobile robotic platforms.

This paper introduces a hybrid framework designed to bridge the gap between intuitive human teaching and autonomous robotic execution. By combining vision-based Learning from Demonstration (LfD) for large-scale "gross" movements with force-based Reinforcement Learning (RL) for delicate "fine" assembly, the authors provide a robust solution for high-precision tasks in uncertain environments.

Overview of the hybrid framework combining LfD for gross motion and RL for fine exploration.

The Bottlenecks of Industrial Assembly

The automation of contact-rich tasks is primarily hindered by two factors:

  1. Programming Complexity: Traditional teaching methods are often impractical. Teach pendants are slow and unintuitive for complex paths. Hand-guiding, while more natural, becomes difficult in "constrained spaces"—cramped factory corners where human access is limited and robot joint limits are easily reached.
  2. Positioning Uncertainty: Even advanced robots face positioning errors, often in the range of ±5 mm. In high-precision assembly where tolerances are sub-millimeter, such errors can lead to excessive contact forces, resulting in part damage or system failure.

A Two-Stage Hybrid Framework

The proposed system addresses these challenges through a bifurcated approach that transitions from visual guidance to tactile exploration.

1. Gross Motion: Contact-Free Visual Servoing

To simplify the teaching process, especially in cramped quarters, the authors introduce "contact-free" teaching. Instead of physically dragging the robot arm, a user guides the robot to an initial grasping pose and then moves the target object toward the goal. An eye-in-hand camera tracks the object, and the robot follows using visual servoing. This method records a demonstrated trajectory without requiring the user to overcome the robot's mechanical resistance or navigate awkward postures.

The experimental setup showing contact-free teaching in a heavily constrained operation space.

2. Fine Motion: Region-limited Residual RL (RRRL)

As the robot approaches the target, it switches to a Region-limited Residual RL policy. This stage focuses on the final millimeters of insertion where vision is often occluded.

  • Residual Learning: The RL agent does not learn from scratch; it uses the human demonstration as a baseline "prior," significantly speeding up convergence.
  • Tactile Feedback: Utilizing 6D force and torque data, a Double DQN policy outputs small, corrective adjustments. This allows the robot to "feel" its way into the socket, wiggling the part into place safely and autonomously.

Experimental Evaluation and Results

The framework was evaluated on a custom Machine Tending Task using a UR5e robot. To simulate real-world conditions, a random positioning error of 2–4 mm was introduced. The proposed method was compared against industry-standard baselines: teach-pendant programming and hand-guiding, both paired with traditional spiral searching.

The results demonstrate significant improvements across all key metrics:

  • Success Rate: The hybrid framework achieved a 91% success rate under uncertainty, vastly outperforming the teach-pendant (47%) and hand-guiding (33%) baselines.
  • Efficiency: Setup time was reduced to 23–30 seconds, compared to 120 seconds for traditional teach-pendant methods.
  • Safety: The maximum contact force during teaching was maintained between 3–10 N, a sharp contrast to the 60 N forces generated during hand-guiding, which pose a risk to delicate hardware.

Conclusion

By decoupling the "where" (vision-based gross motion) from the "how" (force-based fine motion), this research offers a practical path toward more flexible and resilient industrial automation. The "contact-free" teaching approach significantly improves the user experience for non-experts, while the residual RL policy ensures high success rates even in the face of significant positioning errors. Future work will look to incorporate the complex physical properties of grippers, such as suction cup elasticity, directly into the learning models to further enhance precision.

Summary updated

BibTeX
@inproceedings{shi2021combining,
  author = {Yunlei Shi and Zhaopeng Chen and Yansong Wu and Dimitri Henkel and Sebastian Riedel and Hongxu Liu and Qian Feng and Jianwei Zhang},
  title = {{Combining Learning from Demonstration with Learning by Exploration to Facilitate Contact-Rich Tasks}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2021},
  doi = {10.1109/iros51168.2021.9636417},
}
Figure