Introduction
Robotics competitions, such as the European Robotics League (ERL), serve as critical benchmarks for autonomous systems in complex, human-centric environments. However, physical competitions face significant logistical hurdles, including high costs and the need for specialized physical infrastructure like the "Granny Annie" mock apartment. The COVID-19 pandemic further highlighted the vulnerability of these events to travel restrictions and lab closures. To address these challenges, researchers from Heriot-Watt University and the University of Edinburgh developed an automated virtual scoring framework that allows teams to compete remotely using a high-fidelity simulation.
The Virtualized Testbed
The core of the framework is a digital twin of the Robotic Assisted Living Testbed (RALT) located at Heriot-Watt University. This environment is built using the Gazebo simulator and integrated with the Robot Operating System (ROS).
The simulation environment utilizes standard roslaunch files to configure the starting positions of the robot and various interactive objects, ensuring a reproducible and adaptable testing ground for competition tasks. By populating the world with a library of compatible 3D models (food boxes, furniture, etc.), the framework mirrors the complexity of physical service robotics arenas.
System Architecture
The framework introduces a "Virtual Referee" to bridge the gap between simulation and physical competition hardware. In traditional ERL events, a hardware device known as the Referee, Scoring and Benchmarking Box (RSBB) manages communication and task signaling. The researchers recreated the RSBB's functionality entirely in software, maintaining identical ROS topic interfaces. This "zero-code change" approach ensures that robot software developed in simulation can transition to physical hardware without modification.
Automated scoring is achieved by monitoring the simulation's internal state. Rather than relying on human observation, the virtual referee queries Gazebo’s model states to verify task completion. For instance, it can programmatically confirm if a specific object has been moved to a target location, such as delivering a cracker box from a kitchen island to a user.
Implementation and Results
The framework was validated using the "Catering for Granny Annie’s Comfort" scenario. In this task, a robot must receive a command via a virtual tablet, navigate to the kitchen, retrieve a specific item, and deliver it to the user. The system successfully tracked the object's XYZ coordinates and awarded points based on predefined success criteria (e.g., one point for retrieval, one for delivery).
Key findings include:
- Robot Agnosticism: The framework supports various robot platforms provided they use standard ROS interfaces.
- Rapid Iteration: Automation allows for hundreds of test runs in the time required for a single physical setup.
- Robustness Testing: Simulation enables easy modification of layouts and object placements to prevent algorithmic overfitting to a single environment.
Conclusion and Future Directions
While the current framework provides a robust baseline for virtual benchmarking, future work will focus on increasing realism. This includes modeling sensor noise and uncertainty, introducing dynamic obstacles like moving humans to test social navigation, and expanding the library of tasks to encompass the full ERL handbook. By lowering the barrier to entry, this virtual testbed paves the way for more inclusive and frequent robotics competitions.
Summary updated