Care3D: An Active 3D Object Detection Dataset of Real Robotic-Care Environments

Michael G. Adam, Sebastian Eger, Martin Piccolrovazzi, Maged Iskandar, Jörn Vogel, Alexander Dietrich, Seongjien Bien, Jon Škerlj, Abdeldjallil Naceri, Eckehard Steinbach, Alin Albu‐Schäffer, Sami Haddadin, Wolfram Burgard

TL;DR

This paper introduces Care3D, a novel dataset for active 3D object detection in real-world robotic care environments. Addressing the scarcity of realistic training data, Care3D captures diverse indoor scenes and provides detailed annotations, enabling the development of more robust assistive robots for healthcare.

What Problem Does This Paper Address?

The increasing demand for assistive robots in healthcare, driven by labor shortages and an aging population, highlights a critical need for advanced perception capabilities. Specifically, robots need to accurately detect and understand objects in their environment to provide effective assistance. However, developing and evaluating active 3D object detection algorithms, which involve robots strategically moving to gather information, is hampered by a lack of realistic, annotated datasets. Existing datasets are often simulated or capture only static scenes, leading to a "sim-to-real" gap and limiting the applicability of algorithms in dynamic, real-world care settings. This paper aims to bridge this gap by providing a dataset captured in actual healthcare-related environments.

Methods

The core contribution of this paper is the creation and release of the Care3D dataset. The data was captured using a NavVis VLX scanner, which provides high-resolution 360° panoramic images and dense 3D point clouds with centimeter-level accuracy. The scanning process involved capturing images from discrete positions across the environment, mimicking an active exploration strategy. This approach allows for the development of algorithms that can intelligently decide where to move next to gather the most informative data.

The dataset includes two distinct scenarios:

  1. SMiLE Scenario (DLR): This scene represents a test site for assistive robotics in care at the German Aerospace Center (DLR). It includes elements of a small apartment, such as a kitchen, living room, and furniture, designed to simulate everyday activities for people with disabilities.
  2. GARMI Scenario (TUM): This scenario, part of the geriatronics lighthouse project at TUM, features two rooms: one mimicking a doctor's office with tele-operatable robotic arms, and another showcasing the GARMI humanoid robot in a development environment.

For the GARMI scenario, additional sensor data was collected, including RGB-D images from the robot's onboard cameras, IMU data, and LiDAR scans. Crucially, a VICON system was used to capture ground truth pose data for a subset of the GARMI recordings, specifically designed as a test set for visual SLAM algorithms.

Datasets

The paper introduces the Care3D dataset, which comprises two real-world environments:

  • SMiLE Scenario: Captured at the German Aerospace Center (DLR).
  • GARMI Scenario: Captured at the Technical University of Munich (TUM).

Both scenarios feature:

  • 360° panoramic images with 8192x4096 resolution.
  • Dense 3D point clouds with sub-centimeter accuracy.
  • Registered positions of the captured panoramas.
  • 3D bounding box annotations for objects within the scenes.

The GARMI scenario also includes:

  • RGB-D images from the robot's onboard cameras.
  • IMU data.
  • LiDAR data.
  • Ground truth pose data from a VICON system for SLAM evaluation.

The authors note that the dataset follows a similar structure to the atum3d dataset [13], ensuring compatibility with algorithms developed for it.

Key Insights and Ideas

  • The critical need for real-world data: The paper strongly emphasizes that while simulations are useful, they cannot fully replicate the complexities of real-world environments, especially for dynamic tasks like active object detection. Realistic data is essential for bridging the sim-to-real gap.
  • Active perception for robotics: The dataset's capture methodology, involving discrete viewpoints and the potential for intelligent view selection, directly supports the development of active perception strategies for robots. This moves beyond static scene understanding.
  • Dual-purpose dataset: Care3D is not only valuable for 3D object detection but also provides a dedicated SLAM test set with ground truth, enabling comprehensive evaluation of robot localization and mapping capabilities in care environments.
  • Diversity in care environments: The inclusion of both a simulated apartment (SMiLE) and a more clinical/developmental setting (GARMI) offers a broader range of scenarios for training and testing assistive robots.

Key Results

The paper primarily focuses on introducing and describing the dataset. Quantitative results are not presented as this is a dataset paper. However, the dataset's statistics highlight its richness:

  • SMiLE Scenario: Covers 57.24 m² with 58 panoramic images and 40 annotated objects.
  • GARMI Scenario: Covers 138.36 m² across two rooms, with 107 discrete image positions and 70 annotated objects.
  • Object Annotations: 3D bounding boxes with full degree-of-freedom are provided. The average edge length of objects is 0.71 m, and the average volume is 0.45 m³.
  • SLAM Test-Set: Includes synchronized RGB-D, IMU, LiDAR, and VICON ground truth data for evaluating SLAM algorithms.

Limitations and Future Work

  • Limited Class Distribution: The paper acknowledges that the class distribution of the labels is highly nonlinear, with some classes occurring only once. The authors suggest merging these infrequent classes into a single 'other' class.
  • Future Scenarios: The authors explicitly state their intention to add more scenarios to the dataset in the future, further expanding its scope and utility.
  • Active Object Detection Evaluation: While the dataset supports active object detection, the paper itself does not present results from running such algorithms on Care3D. This remains an area for future research.

Summary updated

Abstract

As labor shortage increases in the health sector, the demand for assistive robotics grows. However, the needed test data to develop those robots is scarce, especially for the application of active 3D object detection, where no real data exists at all. This short paper counters this by introducing such an annotated dataset of real environments. The captured environments represent areas which are already in use in the field of robotic health care research. We further provide ground truth data within one room, for assessing SLAM algorithms running directly on a health care robot.
BibTeX
@misc{adam2023care,
  author = {Michael G. Adam and Sebastian Eger and Martin Piccolrovazzi and Maged Iskandar and Jörn Vogel and Alexander Dietrich and Seongjien Bien and Jon Škerlj and Abdeldjallil Naceri and Eckehard Steinbach and Alin Albu‐Schäffer and Sami Haddadin and Wolfram Burgard},
  title = {{Care3D: An Active 3D Object Detection Dataset of Real Robotic-Care Environments}},
  year = {2023},
  eprint = {2310.05600},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2310.05600},
}

Loading related content...

Figure