Robotic Object Insertion with a Soft Wrist through Sim-to-Real Privileged Training

Yuni Fuchioka, Cristian C. Beltran-Hernandez, Hai Nguyen, Masashi Hamaya

Abstract

This study addresses contact-rich object insertion tasks under unstructured environments using a robot with a soft wrist, enabling safe contact interactions. For the unstructured environments, we assume that there are uncertainties in object grasp and hole pose and that the soft wrist pose cannot be directly measured. Recent methods employ learning approaches and force/torque sensors for contact localization; however, they require data collection in the real world. This study proposes a sim-to-real approach using a privileged training strategy. This method has two steps: 1) The teacher policy is trained to complete the task with sensor inputs and ground truth privileged information such as the peg pose, and then 2) the student encoder is trained with data produced from teacher policy rollouts to estimate the privileged information from sensor history. We performed sim-to-real experiments under grasp and hole pose uncertainties. This resulted in 100%, 95%, and 80% success rates for circular peg insertion with 0, +5, and -5 degree peg misalignments, respectively, and start positions randomly shifted ±10 mm from a default position. Also, we tested the proposed method with a square peg that was never seen during training. Additional simulation evaluations revealed that using the privileged strategy improved success rates compared to training with only simulated sensor data. Our results demonstrate the advantage of using sim-to-real privileged training for soft robots, which has the potential to alleviate human engineering efforts for robotic assembly.

Introduction

Precision assembly tasks, such as the classic "peg-in-hole" insertion, remain a significant challenge in robotics when operating in unstructured environments. The primary difficulty stems from uncertainties in the object's grasp and the exact location of the target hole. While rigid robots require high-precision sensing and complex control loops to avoid damage during contact, robots equipped with passive soft wrists offer a mechanical alternative. These compliant wrists allow for safe, passive deformation during contact, naturally accommodating small misalignments.

However, soft wrists introduce a new problem: their deformation is non-linear and difficult to measure directly using standard onboard sensors. Traditionally, overcoming this requires expensive external motion-capture systems or extensive real-world data collection, which is both time-consuming and potentially damaging to the hardware. This research presents a framework that achieves high-precision insertion using a soft wrist while relying exclusively on simulation for training and standard onboard sensors for real-world execution.

Methodology: Privileged Training

The core of this approach is a Sim-to-Real Privileged Training strategy, which decouples the learning of the task logic from the interpretation of noisy sensor data. The process is divided into two distinct phases:

1. Teacher Policy Training

In a MuJoCo-based simulation environment, a "teacher" policy is trained using Reinforcement Learning (specifically, Proximal Policy Optimization or PPO). Because this phase occurs in simulation, the teacher has access to "privileged information"—data that would be unavailable to a real robot. This includes the exact relative pose of the peg and hole, and a binary "alignment signal" indicating if the peg is vertically aligned with the hole. This signal acts as a critical subtask switching condition, helping the policy distinguish between the "searching" and "insertion" phases.

2. Student Policy Training

A "student" agent is then trained to mimic the teacher's behavior. Unlike the teacher, the student only has access to standard sensor data: the robot's arm pose and force/torque readings. The student utilizes a Temporal Convolutional Network (TCN) to analyze a one-second history of this sensor data to estimate the hidden privileged states. By regressing directly onto physical quantities (like peg pose) rather than abstract latent spaces, the system remains interpretable and easier to debug.

Bridging the Sim-to-Real Gap

To ensure the policy transfers successfully to a physical UR5e robot, the researchers employed extensive Domain Randomization. They randomized the peg's in-grasp angle, the hole's position, and the gains of the Operational Space Control (OSC). This randomization accounts for the mechanical discrepancies between a simulated torque-controlled robot and the stiff, position-controlled nature of real-world industrial arms.

Experimental Results

The framework was evaluated using a physical UR5e robot equipped with a custom 3D-printed soft wrist and a Robotiq Hand-E gripper.

Zero-Shot Transfer

The system demonstrated robust performance in the real world without any fine-tuning on physical hardware:

  • Circular Peg (40mm): Achieved a 100% success rate under perfect alignment and maintained high success rates (80-95%) even with ±5° misalignment and ±10mm starting position shifts.
  • Generalization to Unseen Objects: One of the most significant findings was the policy's ability to handle a square peg, an object never encountered during training. The best-performing network achieved a 70% success rate with the square peg, suggesting the RL agent learned fundamental contact-handling strategies rather than simple geometric memorization.

The Role of the Soft Wrist

The study also highlighted a mechanical advantage of the soft wrist: it acts as a natural low-pass filter. The mechanical structure naturally dampened high-frequency torque noise, allowing the control policy to focus primarily on 3D force data without losing critical information.

Conclusion and Future Work

This research demonstrates that privileged training is a powerful tool for controlling soft robotic systems in contact-rich tasks. By leveraging simulation to provide ground-truth data during training, robots can learn to navigate complex physical interactions using only standard sensors.

While successful, the system's primary failure mode occurred when the initial search failed to find the hole's edge, preventing the insertion phase from triggering. Future research will focus on randomizing a wider range of physical parameters—such as surface friction and spring damping—and testing the framework on tighter-tolerance industrial components like electrical connectors to further bridge the gap between simulation and factory-floor application.

Summary updated

Abstract

This study addresses contact-rich object insertion tasks under unstructured environments using a robot with a soft wrist, enabling safe contact interactions. For the unstructured environments, we assume that there are uncertainties in object grasp and hole pose and that the soft wrist pose cannot be directly measured. Recent methods employ learning approaches and force/torque sensors for contact localization; however, they require data collection in the real world. This study proposes a sim-to-real approach using a privileged training strategy. This method has two steps. 1) The teacher policy is trained to complete the task with sensor inputs and ground truth privileged information such as the peg pose, and then 2) the student encoder is trained with data produced from teacher policy rollouts to estimate the privileged information from sensor history. We performed sim-to-real experiments under grasp and hole pose uncertainties. This resulted in 100\%, 95\%, and 80\% success rates for circular peg insertion with 0, +5, and -5 degree peg misalignments, respectively, and start positions randomly shifted $\pm$ 10 mm from a default position. Also, we tested the proposed method with a square peg that was never seen during training. Additional simulation evaluations revealed that using the privileged strategy improved success rates compared to training with only simulated sensor data. Our results demonstrate the advantage of using sim-to-real privileged training for soft robots, which has the potential to alleviate human engineering efforts for robotic assembly.
BibTeX
@inproceedings{fuchioka2024robotic,
  author = {Yuni Fuchioka and Cristian C. Beltran-Hernandez and Hai Nguyen and Masashi Hamaya},
  title = {{Robotic Object Insertion with a Soft Wrist through Sim-to-Real Privileged Training}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=e9HKbLMjY4},
}
Figure