This paper presents an embedded architecture for robotic manipulation that bridges the "reality gap" by combining learned sim-to-real reaching with force-guided analytical placement. By utilizing a compact state-based interface rather than raw pixels, the authors achieve reliable real-world performance on an NVIDIA Jetson platform, maintaining high perception throughput for dynamic control.
TL;DR
The researchers developed a system that splits robotic manipulation into two regimes: learned free-space reaching and force-guided contact correction. By feeding metric, reconstructed targets into compact neural network policies, they achieved a 70% success rate in real-world "sorting" tasks without any real-world fine-tuning, while maintaining a perception latency of just 34.5 ms.
What Problem Does This Paper Address?
Robotic manipulation in the real world faces a triple threat: tight computational constraints on embedded hardware, the "reality gap" when transferring policies from simulation, and the inherent imprecision of vision for contact-rich tasks (like placing a block in a tight hole).
While large-scale visuomotor policies are popular, they often require massive compute and struggle with the subtle physics of contact. Conversely, purely analytical methods can be brittle in unstructured environments. This paper explores how to combine the best of both worlds—learned "reaching" and analytical "placement"—on a shared embedded platform (NVIDIA Jetson AGX Orin) controlling a Franka Research 3 (FR3) robot.
Methods
The core of the approach is a shared embedded front end that reconstructs metric targets from eye-in-hand RGB-D observations.
- Perception: A YOLO11-seg model runs on the Jetson Orin via TensorRT to segment objects. The system then back-projects mask centroids into 3D space using depth data and hand-eye calibration to find the target's position in the robot's base frame.
- The "Bridge" (State-based Interface): Instead of feeding raw images to the reinforcement learning (RL) policy, the authors use a 15-dimensional vector containing joint positions, end-effector pose, object identity, and the reconstructed 3D target. This "externalizes appearance," meaning the RL policy only deals with metric coordinates, significantly reducing domain shift.
- Learned Hybrid Strategy: Uses two Proximal Policy Optimization (PPO) policies trained in the RLBench simulator. Policy A handles object reaching, while Policy B handles coarse hole-region approach. Scripted logic manages the grasping and lifting between these stages.
- Force-Guided Analytical Strategy: A baseline that uses deterministic Cartesian transport followed by a spiral search once contact is detected (~2 N normal force). This allows the robot to "feel" its way into the hole, correcting for visual localization errors.
Datasets
- Simulation: Policies were trained and evaluated using RLBench (built on PyRep and CoppeliaSim). Each policy was tested over 1,000 simulation episodes.
- Perception Training: The YOLO11-seg model was trained on a custom dataset of approximately 100 labeled RGB-D images of the workspace, featuring four classes: Blue circle, Green triangle, Nijntje/Miffy, and Yellow square.
Key Insights and Ideas
- Decoupling Perception from Policy: By converting pixels to metric coordinates before they hit the neural network, the authors bypassed the need for complex domain randomization of textures and lighting. The policy simply sees "where the object is," not "what it looks like."
- The Post-processing Bottleneck: Interestingly, the researchers found that post-processing (mask decoding and centroid extraction) took longer (15.3 ms) than the actual neural network inference (11.1 ms). This highlights that for embedded robotics, optimizing the "glue" code is just as important as optimizing the model.
- Regime-Aware Architecture: The results suggest that while RL is excellent for the "macro" movement of reaching through free space, explicit force-feedback loops are still superior for the "micro" movements of precise placement.
Key Results
The system was evaluated across 100 physical trials (50 per strategy).
- Success Rates: The learned hybrid pipeline achieved a 70.0% completion rate, while the force-guided analytical pipeline achieved 74.0%. The overlapping confidence intervals suggest comparable performance in the evaluated setup.
- Simulation Performance: In the simulator, the PPO policies achieved 87.2% (Policy A) and 83.1% (Policy B) success rates.
- Latency: The total mean perception latency was 34.55 ms, enabling a throughput of approximately 28.9 FPS, which is critical for reactive visual servoing.
Limitations and Future Work
The authors identified three main "vulnerabilities":
- Self-Occlusion: The wrist-mounted camera can lose sight of the object as the gripper gets very close, leading to stale target estimates.
- Perception Jumps: False positives in the segmentation model can cause the target to "jump," leading to discontinuous robot commands.
- Orientation Sensitivity: The current study focused on approximately aligned objects. Future work is needed to handle arbitrary 3D orientations and more complex geometries.
The authors conclude that these results motivate a "regime-aware" architecture that explicitly transitions from learned free-space motion to force-based correction upon contact.
Summary updated