TL;DR
Researchers have developed a robust sim-to-real framework for contact-rich "peg-in-hole" tasks using a robot equipped with a passive soft wrist. By employing a two-stage "privileged training" strategy—where a student agent learns to estimate hidden physical states from sensor history—the system achieves high success rates in the real world without requiring any real-world training data or external motion-capture systems.
What Problem Does This Paper Address?
Precision assembly tasks, like inserting a peg into a hole, are notoriously difficult for robots when there is uncertainty in the object's grasp or the hole's exact location. While "soft" or compliant wrists are excellent for these tasks because they allow for safe, passive deformation during contact, they introduce a major control challenge: the wrist's deformation is nonlinear and cannot be directly measured by standard robot sensors.
Traditionally, researchers solve this by using expensive external motion-capture systems or by collecting massive amounts of interaction data in the real world, which is time-consuming and can damage hardware. This paper tackles the challenge of achieving high-precision insertion using a soft wrist while relying only on simulation for training and standard onboard sensors (arm pose and force/torque data) for execution.
Methods
The authors utilize a Sim-to-Real Privileged Training approach, which breaks the learning process into two distinct phases:
- Teacher Training (RL): A "teacher" policy is trained in a MuJoCo-based simulation using Reinforcement Learning (Proximal Policy Optimization, or PPO). Because it's a simulation, the teacher has access to "privileged information" that a real robot wouldn't have, such as the exact relative pose of the peg and the hole, and a binary "alignment signal."
- Student Training (Imitation): A "student" agent is then trained to mimic the teacher. However, the student does not have access to the privileged information. Instead, it uses a Temporal Convolutional Network (TCN) to analyze a one-second history of standard sensor data (wrist position and force/torque readings) to estimate the privileged states.
To ensure the policy works on a physical robot, the authors used extensive Domain Randomization. They randomized the peg's in-grasp angle, the hole's position, and—crucially—the gains of the Operational Space Control (OSC) to account for the mechanical differences between a simulated torque-controlled robot and a stiff, position-controlled real-world UR5e.
Datasets
As an RL-based study, the "dataset" consists of the simulation environment and the physical evaluation setup:
- Simulation: Built on Robosuite and MuJoCo, featuring a custom-modeled soft wrist consisting of four springs.
- Physical Evaluation: Conducted on a Universal Robots UR5e with a Robotiq Hand-E gripper and a custom 3D-printed soft wrist.
- Objects: Training was performed exclusively on a circular peg (40mm diameter). Testing included the circular peg and an unseen square peg to evaluate generalization.
Key Insights and Ideas
- The "Alignment Signal" is Critical: Including a binary signal in the privileged information that tells the teacher if the peg is vertically aligned with the hole proved vital. It acts as a subtask switching condition, helping the policy learn when to transition from "searching" for the hole to "inserting" into it.
- Estimating Physics vs. Latent Spaces: Unlike some prior works that use abstract "latent embeddings," this student encoder directly regresses onto physical quantities (like peg pose). This makes the system more interpretable and easier to debug.
- Soft Wrists as Low-Pass Filters: The authors observed that the soft wrist's mechanical structure naturally filtered out high-frequency torque noise, allowing them to ignore torque readings and focus solely on 3D force data for the control policy.
- Zero-Shot Generalization: The policy learned on a circular peg generalized surprisingly well to a square peg, suggesting the RL agent learned fundamental contact-handling strategies rather than just memorizing a specific geometry.
Key Results
The system demonstrated impressive zero-shot transfer from simulation to the real world:
- Circular Peg: Achieved a 100% success rate with 0° misalignment and an 80-95% success rate with ±5° misalignment, even when the starting position was shifted by ±10mm.
- Square Peg (Unseen): Despite never seeing a square peg during training, the "best" network achieved a 70% success rate in the 0° misalignment condition.
- Simulation Advantage: In head-to-head sim tests, the privileged training approach significantly outperformed a baseline TCN that tried to learn the task directly from sensor data without the teacher-student decomposition.
Limitations and Future Work
The primary failure mode observed was the peg "missing" the hole entirely. Because the policy only attempts insertion after detecting a specific contact impact, if the initial search fails to find the hole's edge, the robot may never trigger the insertion phase.
The authors suggest that future work could involve:
- Randomizing more physical parameters like surface friction and spring damping to further close the sim-to-real gap.
- Adding reward penalties to encourage faster task completion.
- Testing the system on more complex, tighter-tolerance industrial components like electrical connectors.
Summary updated