In the field of robotics, learning the outcomes of symbolic actions through real-world experimentation is often prohibitively risky or expensive. A robot attempting to learn how to dismantle a device might inadvertently damage itself or the target object during the exploration phase. This paper, presented at IROS 2020, addresses this challenge by introducing a framework that leverages multiple simulation environments of varying fidelity to accelerate learning while minimizing real-world failures.
The MENID Framework
The core contribution of this work is the MENID (Multiple Environment Noise Indeterministic Deictic) rule. This framework extends standard probabilistic planning rules to track action outcomes across multiple environments simultaneously. By doing so, a robot can utilize "cheap" but potentially inaccurate low-fidelity simulators to build an initial model before graduating to a "dear" high-fidelity target environment.
As shown in the architecture above, the system incorporates an Environment Selector that decides whether to perform an action in a fast simulation or the target environment. This decision is governed by a confidence-based algorithm. The robot uses Dirichlet sampling to estimate the error of its current success probability distribution; if the variance is high, it remains in simulation. Once a confidence threshold ($\delta$) is met, the robot transitions to the target environment.
Bridging the Fidelity Gap
A critical challenge in multi-environment learning is ensuring that simulation errors do not lead to real-world failures. The authors implement a Decreasing-m Estimator, which acts as a mathematical bridge. Initially, the robot's internal model relies heavily on simulation data. However, as real-world data is collected, the influence of the simulator is automatically phased out, ensuring the final planning model is grounded in physical reality.
Experimental Validation: Hard Drive Dismantling
The approach was evaluated using a complex dismantling task involving a hard drive. The robot had to learn the success probabilities for three distinct actions: levering a PCB, shaking the drive bay, and using a suction cup. The study utilized the SOFA Framework for physics simulation, allowing for the modeling of both rigid and deformable objects.
One of the most significant findings is the concept of the "Good Enough" Simulator. The researchers generated 36 different mesh qualities for the hard drive components to study the impact of resolution. They discovered that simulators do not need perfect geometric accuracy to be symbolically useful. As seen in the results above, there is a "sweet spot" where increasing mesh resolution yields diminishing returns for symbolic success, allowing robots to learn effectively even with simplified models.
Key Results and Insights
- Faster Convergence: The interleaved learning approach reached high reward levels significantly faster than baselines that relied solely on the target environment.
- Failure Avoidance: By the time the robot moved to the physical system (a WAM robotic arm), it had already developed a reliable "hunch" about which actions were likely to succeed, effectively avoiding the accumulation of failures typical in early-stage learning.
- Risk-Awareness: The benefits of this multi-tier approach scale with the penalty of failure. In high-stakes environments where breaking a part is costly, the value of safe, virtual pre-testing becomes paramount.
Conclusion and Future Work
The MENID framework demonstrates that low-fidelity simulations, despite their noise, are powerful tools for bootstrapping robotic intelligence. Future work aims to move beyond learning probabilities for pre-defined rules toward learning the structure of the rules themselves, potentially allowing robots to discover entirely new manipulation strategies autonomously.
Summary updated