Abstract
ROBEL is an open-source platform of cost-effective robots designed for reinforcement learning in the real world. ROBEL introduces two robots, each aimed to accelerate reinforcement learning research in different task domains: D'Claw is a three-fingered hand robot that facilitates learning dexterous manipulation tasks, and D'Kitty is a four-legged robot that facilitates learning agile legged locomotion tasks. These low-cost, modular robots are easy to maintain and are robust enough to sustain on-hardware reinforcement learning from scratch with over 14000 training hours registered on them to date. To leverage this platform, we propose an extensible set of continuous control benchmark tasks for each robot. These tasks feature dense and sparse task objectives, and additionally introduce score metrics as hardware-safety. We provide benchmark scores on an initial set of tasks using a variety of learning-based methods. Furthermore, we show that these results can be replicated across copies of the robots located in different institutions.
Introduction
Reinforcement Learning (RL) has achieved significant success in simulated environments, yet the transition to physical hardware—the "sim-to-real" gap—remains a major barrier. Simulations often fail to capture complex physical phenomena like friction and contact dynamics. While training directly on hardware is ideal, it is often hindered by high costs, the fragility of industrial robots, and a lack of standardized benchmarks. ROBEL (Robotics Benchmarks for Learning) addresses these issues by providing an open-source platform of cost-effective, modular robots designed specifically for real-world RL.
Methods: The ROBEL Platform
The ROBEL platform integrates hardware, software, and safety-focused benchmarking. The hardware consists of two primary robots built from off-the-shelf components and 3D-printed parts:
- D’Claw: A 9-DOF three-fingered hand designed for dexterous manipulation tasks.
- D’Kitty: A 12-DOF quadruped robot for agile legged locomotion.
The software component provides a Python library following the OpenAI Gym API, facilitating a unified interface for both physical hardware and simulated back-ends (MuJoCo/Bullet). To ensure hardware longevity during the "trial-and-error" phase of RL, ROBEL introduces hardware-safety metrics ($\phi_{hs}$) that monitor joint limits, velocities, and torque spikes, encouraging the development of safer algorithms.
Benchmarks and Tasks
ROBEL provides a suite of continuous control tasks categorized by difficulty:
- Manipulation (D’Claw): Tasks include Pose (matching joint angles), Turn (rotating a valve), and Screw (continuous object rotation).
- Locomotion (D’Kitty): Tasks include Stand (upright positioning), Orient (heading alignment), and Walk (target reaching).
These tasks are offered in variants—Fixed, Random, and RandomDynamics—to evaluate generalization and robustness to physical variations like friction and mass.
Key Results and Insights
A primary advantage of ROBEL is its cost-effectiveness. D’Claw and D’Kitty are significantly more affordable than industrial alternatives, lowering the barrier to entry for real-world robotics research. D’Claw costs approximately $3,500 and D’Kitty $4,200, making them accessible to academic labs.
The platform's robustness is evidenced by over 14,000 cumulative training hours registered across multiple sites. Despite the intensity of RL exploration, the robots required minimal maintenance. Furthermore, the authors demonstrated reproducibility; robots built at different institutions using the same documentation exhibited nearly identical learning performance and could share trained policies.
Conclusion
ROBEL successfully brings RL out of simulation by providing a hardy, low-cost, and standardized hardware platform. While challenges remain in handling extreme environmental uncertainty, ROBEL provides the necessary foundation for the community to develop safer and more robust robotic controllers. The platform is fully open-source, encouraging further contributions to hardware and task design at www.roboticsbenchmarks.org.
Summary updated