Abstract
Table-top Rearrangement and Planning (TTRP) is a fundamental challenge in domestic robotics, requiring systems to autonomously organize cluttered environments. This paper presents a critical analysis of the perception stack utilized during the ICRA 2022 OCRTOC challenge. We identify significant bottlenecks in scene registration, 6D pose estimation, and grasp selection, particularly when dealing with unknown objects and sensor noise. Our findings suggest that while current model-free approaches are promising, substantial improvements in computational efficiency and structural understanding are required for real-world deployment.
Introduction
The goal of TTRP is to enable a robot to perceive a disordered scene and rearrange objects into a target configuration. While motion planning is relatively mature, the perception of unknown objects in clutter remains a major hurdle. Robots in novel environments often encounter objects not present in their training sets, necessitating robust generalization. This work evaluates the standard perception pipeline against the constraints of the ICRA 2022 Open Cloud Robot Table Organization Challenge (OCRTOC), where the authors secured a third-place finish.
Methodology
The standard TTRP perception pipeline is generally decomposed into three distinct stages:
- Scene Registration: This involves capturing 3D point clouds from multiple viewpoints and stitching them into a coherent global frame, typically using Iterative Closest Point (ICP) algorithms.
- 6D Object Pose Estimation: The system determines the precise position and orientation of objects. Indirect methods are often employed, matching 2D RGB projections to 3D models via deep learning descriptors (e.g., ResNet-based features).
- Manipulation Point Detection: For unknown objects, "model-free" grasp generators like GraspNet-1Billion or Contact-GraspNet are used to propose grasp candidates based solely on point cloud geometry.
Critical Bottlenecks and Insights
Our analysis reveals several key challenges that hinder performance in competitive and real-world settings:
- Latency: Current perception stacks are computationally expensive. Scene registration can take approximately 1.5 minutes, while full-scene pose estimation may exceed 5 minutes. This latency is a primary barrier to fluid human-robot interaction.
- The "Hole" Problem: Sensor noise and occlusions often result in incomplete point clouds. Grasping algorithms frequently fail by proposing grasp points within these "holes," leading to inevitable collisions or missed grasps.
- Structure-Agnostic Grasping: Model-free methods often lack semantic understanding. A robot might attempt to grasp a tool by its functional pivot rather than its handle because it treats all geometry as equivalent.
Conclusion and Future Directions
To overcome these limitations, we propose several research directions. First, reducing the dependency on 3D CAD models is essential for "in the wild" operation. Second, shifting from slow 3D registration to direct 3D grasp prediction from 2D RGB images could significantly reduce latency. Finally, integrating tactile feedback and reinforcement learning for grasp ranking could provide the necessary robustness to handle sensor noise and functional structural requirements that purely geometric methods currently miss.
Summary updated