Flexible Gear Assembly With Visual Servoing and Force Feedback

Junjie Ming, Daniel Bargmann, Hongpeng Cao, Marco Caccamo

Abstract

Gear assembly is an essential but challenging task in industrial automation. This paper presents a novel two-stage approach for achieving high-precision and flexible gear assembly. The proposed approach integrates YOLO to coarsely localize the workpiece in a searching phase and deep reinforcement learning (DRL) to complete the insertion. Specifically, DRL addresses the challenge of partial visibility when the on-wrist camera is too close to the workpiece. Additionally, force feedback is used to smoothly transit the process from the first phase to the second phase. To reduce the data collection effort for training deep neural networks, we use synthetic RGB images for training YOLO and construct an offline interaction environment leveraging sampled real-world data for training DRL agents. We evaluate the proposed approach in a gear assembly experiment with a precision tolerance of 0.3mm. The results show that our method can robustly and efficiently complete searching and insertion from arbitrary positions within an average of 15 seconds.

Introduction

In industrial automation, gear assembly represents a classic high-precision challenge. Traditional robotic methods often rely on "position teaching," where a human operator manually programs every coordinate. However, this approach is brittle; even slight shifts in the workpiece's position can lead to system failure. While vision-based and force-based systems offer more flexibility, they encounter a significant "blind spot": as the robot arm approaches the target for insertion, the on-wrist camera often loses a full view of the workpiece, leading to partial visibility issues. Furthermore, mapping complex force/torque signals to precise movements remains mathematically difficult.

Experimental setup featuring a Franka Emika Panda robot arm and the gear assembly platform.

This research introduces a robust two-stage pipeline that combines computer vision for coarse localization with Deep Reinforcement Learning (DRL) for fine insertion, achieving a precision tolerance of 0.3 mm.

Methodology

The proposed system operates in two distinct phases: searching and insertion.

The robot utilizes an on-wrist RGBD camera and the YOLO (You Only Look Once) algorithm to locate the target platform. To minimize the need for manual data labeling, YOLO is trained entirely on synthetic images generated in Blender. Using depth data, the system calculates 3D coordinates and moves the robot to the general vicinity of the assembly peg.

Fine Alignment and Insertion

Once the robot detects contact via a force sensor, it transitions from visual searching to a DRL-based controller. This transition is managed by force feedback, which acts as a robust trigger for the second phase.

The fine alignment phase addresses the offset 'e' between the gear and the peg under partial visibility.

To handle partial visibility, the DRL agent is trained to infer the relative position of the peg's center by analyzing the edges and textures of the partially visible target platform. The authors compared two DRL algorithms: DQN (Discrete actions) and PPO (Continuous actions). PPO was found to be superior, as its continuous action space allows for smoother, more precise movements that compensate for calibration errors.

To ensure data efficiency, the DRL agent was trained in an offline interaction environment. This "grid world" was constructed from real-world images and coordinates sampled at 1mm intervals, allowing the agent to practice extensively before deployment on the physical robot.

Key Results and Insights

The system was evaluated using a Franka Emika Panda robot across 100 randomly sampled starting positions. Key findings include:

  • High Reliability: The fine-tuned PPO model achieved a 100% success rate.
  • Industrial Precision: The system successfully managed a 0.3 mm tolerance, meeting standard industrial requirements for gear fits.
  • Efficiency: The entire pipeline completes the task in an average of 15 seconds, with the DRL-guided insertion phase requiring only about 6 seconds. The PPO agent also demonstrated high path efficiency, with a Traveling Score (ATS) of 1.3.
  • Visual Landscape Learning: A critical insight is the DRL's ability to navigate based on partial views. By training on images where the peg is off-center, the robot learns to "know" the center's location even without a direct line of sight.

Conclusion and Future Work

While the current approach is highly effective for 3D space (X, Y, and Z-axis insertion), it relies on visually distinguishable spatial features. Future work will focus on extending the pipeline to handle 6 Degrees of Freedom (including complex rotations and tilting) and integrating vision-based grasping to achieve full autonomy from the initial tray pick-up to final assembly.

Summary updated

Abstract

Gear assembly is an essential but challenging task in industrial automation. This paper presents a novel two-stage approach for achieving high-precision and flexible gear assembly. The proposed approach integrates YOLO to coarsely localize the workpiece in a searching phase and deep reinforcement learning (DRL) to complete the insertion. Specifically, DRL addresses the challenge of partial visibility when the on-wrist camera is too close to the workpiece. Additionally, force feedback is used to smoothly transit the process from the first phase to the second phase. To reduce the data collection effort for training deep neural networks, we use synthetic RGB images for training YOLO and construct an offline interaction environment leveraging sampled real-world data for training DRL agents. We evaluate the proposed approach in a gear assembly experiment with a precision tolerance of 0.3mm. The results show that our method can robustly and efficiently complete searching and insertion from arbitrary positions within an average of 15 seconds.
BibTeX
@inproceedings{ming2023flexible,
  author = {Junjie Ming and Daniel Bargmann and Hongpeng Cao and Marco Caccamo},
  title = {{Flexible Gear Assembly With Visual Servoing and Force Feedback}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2023},
  doi = {10.1109/iros55552.2023.10341833},
}
Figure