Introduction
In industrial robotics, "insertion" tasks—such as plugging a USB-C cable or fitting specialized connectors—are notoriously difficult. These tasks are highly sensitive to geometry, lighting, and positioning. Traditional industrial robots rely on high-precision localization, which often fails in unstructured environments like messy factory floors where exact socket coordinates are unknown.
While Reinforcement Learning (RL) allows robots to learn from experience, it typically requires a manually defined "reward function" for every new object. To overcome this bottleneck, researchers have developed Learning on the Job (LOTJ), a system that enables robots to master never-before-seen connectors using only visual input and self-generated rewards.
The LOTJ Framework
The LOTJ system follows a four-phase pipeline designed to transition a robot from general knowledge to task-specific mastery in approximately 50 minutes.
- Offline Pretraining: The process begins with a large-scale dataset of 70,000 trajectories involving 50 different connector types, ranging from standard USBs to specialized industrial NEMA plugs.
- Self-Rewarding System: A reward classifier is trained to recognize successful insertions from camera feeds. This exploits the insight that recognizing success is significantly easier for a model than performing the task itself.
- Domain Adversarial Information Bottleneck (DAIB): This novel representation learning technique splits visual features into domain-invariant components (general shapes) and domain-specific components (unique connector details), preventing the model from overfitting to irrelevant background details.
- Online Finetuning: When presented with a new connector, the robot uses its pretrained policy to attempt the task, grades itself using the reward classifier, and updates its behavior via Implicit Q-Learning (IQL).
Technical Innovation: DAIB
A core challenge in cross-domain robotics is preventing the model from "cheating" by focusing on the robot's gripper or the table color. The DAIB architecture addresses this by using an adversarial loss to force the domain-invariant representation ($z_I$) to "forget" the specific connector identity. Simultaneously, the domain-specific representation ($z_S$) is passed through an information bottleneck to filter out noise, ensuring the policy focuses only on semantically meaningful relationships between the connector and the socket.
Experimental Results
The system was evaluated using two 7-DoF Sawyer robots on "held-out" connectors never seen during training.
Success Rates and Robustness
In scenarios with ±10mm of positioning error, LOTJ achieved a success rate exceeding 75%, significantly outperforming standard RL methods like SAC. In even more challenging "offset" scenarios (10-20mm error), where the initial offline policy often had a 0% success rate, online finetuning allowed the robot to reach success rates as high as 95%.
Visual Attention
One of the most compelling results of the finetuning process is the shift in the robot's visual attention. Grad-CAM visualizations demonstrate that as the robot learns, its internal "focus" moves from uninformative regions to the critical contact point between the connector and the socket.
Conclusion and Future Work
LOTJ demonstrates that robots can autonomously adapt to new, complex manipulation tasks in under an hour. However, the system currently relies heavily on the accuracy of the reward model. If the classifier produces false positives, the robot may learn incorrect behaviors. Future research will explore incorporating active learning—where the robot asks for human help when uncertain—and lifelong learning to continuously expand the robot's global knowledge base.
Summary updated