Small Object Navigation with Context Information

Jiaming Wang, Harold SohAAAI 2023 · DOI

TL;DR

This paper tackles the challenge of robots navigating and finding small household objects, which are often missed by standard object detectors. They introduce a novel probabilistic semantic mapping approach that leverages contextual information to infer object presence, significantly improving navigation success rates.

What Problem Does This Paper Address?

Robots are increasingly expected to perform tasks in human environments, which often involves finding specific objects. A common approach is to use Simultaneous Localization and Mapping (SLAM) to build a semantic map of the environment. This typically involves an object detection module that labels points in a 3D map. However, existing object detectors struggle with small objects like mugs or bowls due to their size and potential occlusion, leading to failures in the entire navigation pipeline. This paper addresses this critical gap by developing a system that can reliably navigate to and find these small, everyday objects.

Methods

The core of the proposed solution is a probabilistic semantic map that incorporates contextual information. Instead of relying solely on direct object detection, their system uses a trained mapping module that estimates the likelihood of an object's presence based on its surroundings.

Here's a breakdown of their approach:

  • Probabilistic Semantic Map: This map stores the probability of different object classes being present in specific locations. It's updated using a differentiable spatial transformation mechanism.
  • Contextual Information Integration: The mapping module uses both visual (RGBD) and contextual information. For instance, it learns that a mug is more likely to be found on a table than on the floor. This is achieved by a neural network that takes RGBD inputs and a "local map" (representing the agent's immediate surroundings) to predict the ego-centric map.
  • Bayesian Updates: The probabilistic map is updated using approximate Bayesian inference, specifically in the logit space for computational efficiency.
  • Normalized BCE Loss: To prevent the network from being dominated by larger objects, they use a normalized Binary Cross-Entropy (BCE) loss that accounts for the size of objects in the local map, ensuring that smaller objects receive adequate training signals.
  • Priority Sampling: For efficient training, they employ a priority sampling technique that focuses on configurations where the agent previously made errors, storing and re-sampling from a buffer of calculated BCE losses.
  • Neural SLAM Integration: The system also incorporates a neural SLAM module to predict local obstacle maps, which are used to update a global obstacle map. This obstacle map, along with the probabilistic semantic map, guides the agent's high-level goal selection and low-level actions.

Datasets

The experiments were conducted using the iGibson scene dataset. To address the lack of diversity in small objects within the original dataset, the authors enhanced the 3D scenes by randomly introducing common household objects like books, bowls, and mugs. They generated 8000 unique training episodes and 200 testing episodes using two novel layouts.

Key Insights and Ideas

  • Context is King for Small Objects: The most significant insight is that contextual cues are crucial for robots to infer the presence of small objects, especially when direct detection fails. Humans do this naturally, and this work demonstrates its effectiveness in a robotic navigation context.
  • Probabilistic Mapping for Uncertainty: Moving beyond deterministic object detection to a probabilistic representation of object presence allows the agent to handle uncertainty more effectively and explore more intelligently.
  • Tailored Loss Functions Matter: The normalized BCE loss is a clever way to ensure that the training process doesn't ignore small objects, which is a common pitfall in object detection tasks.
  • Small Objects are a Distinct Challenge: The results clearly show that small object navigation is significantly harder than navigating to larger objects, highlighting the need for specialized approaches.

Key Results

The proposed method achieved a 26% absolute increase in success rate compared to a strong baseline (Neural SLAM with Frontier-Based Exploration) on the Small Object Navigation (SONav) task. Furthermore, the SPL (Success weighted by Path Length) metric saw a substantial increase of +105%, indicating that the agent not only found the objects more often but also did so more efficiently. Qualitative results (Figure 4) demonstrate the system's ability to infer object presence based on context, even when direct object detection fails.

Limitations and Future Work

  • Limitations: The paper acknowledges that the success of the system heavily relies on the quality of the pre-trained object detection module, even though it's not the sole determinant. The task is also limited to specific small object classes.
  • Future Work: The authors suggest that learning a more efficient exploration policy is a promising direction for future research. Additionally, extending the system to handle a wider variety of objects and environments would be valuable.

Summary updated

Abstract

We introduce a novel framework designed to effectively address the object goal navigation task, specifically focusing on smaller daily household objects such as bowls or mugs. These diminutive objects present challenges to existing SLAM-based semantic mapping methods, as the object detection module employed in the mapping pipeline struggles to accurately detect them. To address this limitation, we propose to use a probabilistic semantic map. It is updated by a trained mapping module that incorporates contextual information to estimate the likelihood of object presence within the agent's current field of view. Subsequently, this probabilistic map guides the agent towards more promising areas for object search. Our experimental results demonstrate that the proposed method surpasses the SOTA approach by 26% in Small Object Navigation tasks.
BibTeX
@inproceedings{wang2023small,
  author = {Jiaming Wang and Harold Soh},
  title = {{Small Object Navigation with Context Information}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaaiss.v1i1.27487},
}
Figure