Abstract
Transferring robotic skills from simulation to the real world remains a significant hurdle due to discrepancies in sensing and actuation. In the ICRA 2024 Robotic Sim2Real (RSC) Challenge, the "ZeroBug" team demonstrated a robust engineering approach to bridge this gap. By implementing a motion-blur-resistant visual pipeline and a feedback-linearized servo system, the team achieved a 100% success rate in real-world long-horizon pick-and-place tasks, securing first place in the competition. This paper details their modular strategy, which prioritizes classical control and targeted perception over complex reinforcement learning.
1. Introduction
The "sim-to-real gap" is a persistent challenge in robotics, where policies trained in high-fidelity simulators often fail when deployed on physical hardware. These failures typically stem from two primary sources: sensing discrepancies, such as motion blur and lighting variations, and actuation discrepancies, including motor dead zones and friction. While Reinforcement Learning (RL) with domain randomization is a popular research direction, it often requires extensive data and can be unpredictable. This work presents an alternative: a modular, principled engineering framework that achieves high-performance transfer without environment-specific code modifications.
2. Addressing Sensing Discrepancies: SMMS
Mobile robots frequently encounter motion blur, which significantly degrades the performance of standard visual markers like ArUco. To counter this, the authors developed the Sequential Motion-blur Mitigation Strategy (SMMS).
2.1 Image Enhancement and Classification
The perception pipeline begins with targeted image processing, boosting the red channel and subtracting the green channel to isolate the "mineral" markers used in the competition. For classification, the team replaced traditional template matching with a lightweight CNN (modeled after LeNet-5). This CNN achieved inference speeds of approximately 11.14 ms, allowing for real-time reactive control.
2.2 Robustness and Filtering
To ensure reliability, the system incorporates a recognition rejection threshold to filter false positives. Furthermore, a low-pass filter combined with extrapolation techniques allows the robot to maintain a "lock" on targets even during frames where the marker is severely blurred or temporarily lost. This approach increased real-world recall from 50% to 70%.
3. Robust Control via Feedback Linearization
Actuation discrepancies, particularly motor non-linearities and dead zones, often lead to steady-state errors in real-world deployment. The authors addressed this using a feedback-linearized servo system for the robot's omnidirectional chassis.
3.1 Design Functions (DF)
The team evaluated three distinct Design Functions to determine the optimal error attenuation strategy:
- Type I (Linear): A standard linear approach, which proved sensitive to motor dead zones.
- Type II (Power-law): Designed to boost output at small error margins, yet still lacked sufficient precision.
- Type III (Integral): This function incorporated an integral term to eliminate steady-state error.
The Type III controller emerged as the most robust, achieving sub-centimeter accuracy (0.81 cm in x, 0.63 cm in y) in the physical arena.
4. Experimental Results
The system was validated within the 2024 Robotic Sim2Real Challenge framework, utilizing the Habitat simulator for training and a physical EP robot for real-world testing.
4.1 Performance Metrics
The perception pipeline's efficiency (11.14 ms/frame) enabled high-speed operation. In the final competition phase, the robot demonstrated exceptional reliability, achieving a 100% success rate (6/6) for both grasping and three-layer stacking in the real world.
4.2 The "Penetration" Paradox
An interesting finding during testing was the "penetration" paradox. Simulators often allow grippers to slightly penetrate object meshes, leading to successful grasps in sim that would cause physical collisions and failures in reality. The high-precision Type III servo system mitigated this by ensuring near-perfect alignment, preventing "diagonal" grasp failures.
5. Conclusion and Future Work
The ZeroBug team's success at ICRA 2024 underscores the efficacy of modular engineering in solving sim-to-real challenges. By specifically targeting motion blur and actuation non-linearities, they achieved superior performance compared to many black-box learning methods. Future work will focus on generalizing the perception pipeline beyond color-specific markers and adapting the system for high-speed, unstructured industrial environments.
Summary updated