ORIon-UTBMan 2023 Team Description Paper

Samuel Sze, Ricardo Cannizzaro, Matthew Munks, Kim Tien Ly, Ana Deligny, Arundathi Shaji Shanthini, Lars Kunze, Ioannis Havoutis, Nick Hawes, Victor Pozo-Fernandez, Iaroslav Okunevich, Zhi YanSPIRE 2023 · OpenAlex

Abstract

The ORIon-UTBMan team, a collaboration between Oxford University and UTBM France, presents a robust software architecture for the Toyota Human Support Robot (HSR) competing in the 2023 RoboCup@Home Domestic Standard Platform League (DSPL). By integrating state-of-the-art open-source methods for object detection, grasp synthesis, and human tracking into a highly modular ROS-based system, the team provides a blueprint for autonomous service robotics in dynamic domestic environments.

Introduction

Operating an autonomous service robot in unstructured domestic settings requires solving multiple complex challenges simultaneously. The robot must perceive its surroundings, manipulate diverse objects, understand human commands, and navigate safely around people. This paper addresses the systems engineering and algorithmic challenge of integrating these disparate capabilities into a cohesive, reliable software stack for the standard Toyota HSR platform.

Furthermore, the project tackles the logistical challenge of building a unified system across a cross-country joint team. By emphasizing high modularity and standardized ROS communication interfaces, the architecture ensures that individual subsystems can be developed independently without compromising the overall robot behavior.

System Architecture and Methods

The team employs a modular, ROS-based architecture governed by hierarchical SMACH state machines for task-level planning. Key subsystem approaches include:

Perception and Semantic Mapping

The perception pipeline utilizes a Faster R-CNN with a ResNet-50-FPN backbone for object detection, pre-trained on the COCO dataset. Detections are stored in a MongoDB-backed semantic map, allowing the robot to evaluate spatial relationships (e.g., "is object A above object B?") and maintain a persistent memory of its environment.

Manipulation Pipeline

For manipulation, the system utilizes Grasp Pose Detection (GPD) on filtered point clouds to synthesize 6-DOF grasp poses. To handle edge cases where standard point-cloud-based synthesis is unreliable—such as small, flat objects like cutlery—the team implemented a custom visual feedback loop. This system uses the in-hand camera to align the end-effector from above and can dynamically switch between the gripper and a vacuum pad based on the object's properties.

The manipulation pipeline integrates point cloud filtering, grasp pose synthesis, and occupancy mapping to generate collision-free joint motion.

Human-Robot Interaction (HRI)

The HRI module combines MediaPipe and OpenPose for 2D skeleton tracking, projecting keypoints into 3D space using RGB-D camera intrinsics. This approach avoids computationally heavy native 3D pose estimation while enabling efficient real-time spatial tracking. Gestures are classified using a 3-layer Long Short-Term Memory (LSTM) neural network trained on a custom dataset of temporal sequences.

The 3-layer LSTM architecture used for classifying human gestures from temporal sequences of keypoints.

Navigation is handled via a probabilistic topological map for global routing. The system implements social navigation behaviors, such as maintaining appropriate social distance, facing humans during interaction, and overtaking on the left.

Key Insights

  • Strict Modularity: Defining clear communication structures (ROS topics and services) between modules allowed the joint UK-France team to integrate a complex system without micromanaging underlying code implementations.
  • Hybrid Manipulation Strategies: Standard point-cloud-based grasp synthesis often fails on flat objects due to poor sensor representation. A custom top-down visual alignment system is necessary for reliable domestic assistance.
  • Efficient 3D Tracking: Extracting 2D keypoints and projecting them into 3D space provides a high-performance alternative to native 3D pose estimation models.

Results and Future Work

The custom LSTM gesture recognition model achieved high categorical accuracy and successfully generalized to unseen data after 100 training epochs. While this paper serves as a pre-competition description, the architecture builds on a strong pedigree, with the constituent teams previously securing 6th place at RoboCup@Home 2019 and 2nd place at the World Robot Challenge 2021.

Future work will address current limitations, specifically target loss during multi-tasking. The team plans to implement an active target recovery system that remembers the last known spatial position of a human and uses voice interaction to request the user to return to the robot's field of view. Additionally, efforts will focus on improving local path planning for dynamic obstacle avoidance and correcting localization drifts.

Summary updated

Abstract

RoboCup 2023
BibTeX
@inproceedings{sze2023orion,
  author = {Samuel Sze and Ricardo Cannizzaro and Matthew Munks and Kim Tien Ly and Ana Deligny and Arundathi Shaji Shanthini and Lars Kunze and Ioannis Havoutis and Nick Hawes and Victor Pozo-Fernandez and Iaroslav Okunevich and Zhi Yan},
  title = {{ORIon-UTBMan 2023 Team Description Paper}},
  booktitle = {SPIRE},
  year = {2023},
}
Figure