Building a grid-semantic map for the navigation of service robots through human-robot interaction

Henry Cheng Zhao, Cheng Zhao 0002, Weixing Mei, Wei PanDigit. Commun. Networks 2015 · DOI

TL;DR

This paper introduces an interactive system for service robots to build "grid-semantic maps" by combining visual and voice input from humans. This allows robots to navigate using intuitive human language, overcoming the limitations of purely geometric maps and making them more accessible for everyday use. A novel localization method is also proposed to improve the robot's ability to determine its position within these maps.

What Problem Does This Paper Address?

Traditional service robots rely on geometric maps for navigation, which are often difficult for the general public to use or create. Humans, on the other hand, naturally navigate using semantic information (e.g., "go to the kitchen"). This paper addresses the challenge of enabling service robots to build and use these intuitive semantic maps through human-robot interaction. Specifically, it tackles how to effectively integrate human voice and visual cues with robot sensor data to create a map that is both geometrically accurate and semantically rich, and how to ensure the robot can reliably locate itself within this map.

Methods

The proposed system is built on the Robot Operating System (ROS) framework and comprises four modules: Interactive, Control, Navigation, and Mapping.

  • Interactive Module: This module handles human input.
  • Visual Interaction: Uses Kinect-based skeleton tracking to identify and follow users, even in the presence of interference. It translates specific body gestures (e.g., raising a hand) into commands for the robot.
  • Vocal Interaction: Leverages the IFLYTEC cloud interface for voice recognition and the Ekho library for voice synthesis. Users can issue commands like "begin mapping" or "go to office 506" via a smartphone app. The system uses Augmented Backus-Naur Form (ABNF) files to parse these commands and provides voice feedback for confirmation.
  • Mapping Module:
    • Grid Map: Employs the GMapping package (based on Rao-Blackwellized Particle Filter - RBPF) to build a real-time indoor grid map using laser scanner and odometer data.
  • Grid-Semantic Map: This is the core contribution. During the mapping process, users can label locations using voice commands. The system combines the semantic name (e.g., "Office 506") with the robot's coordinate data from the grid map to create semantic markers. These markers are stored, forming a Grid-Semantic map that provides a common descriptive language for both the robot and humans.
  • Navigation Module:
    • Global Localization: A novel "localization-evaluate-relocalization" method is proposed. This method uses a modified maximum particle weight within an Adaptive Particle Filter to improve the accuracy and robustness of determining the robot's position on the map. It involves assessing particle swarm convergence and then evaluating the maximum particle weight for successful localization.
  • Path Planning: Once localized and given a destination (via voice command), the system transforms the Grid-Semantic map into a Cost-Map (with an inflation process for obstacles). It then uses Dijkstra's algorithm to find the shortest path and a Path Tracking algorithm to execute it.
  • Control Module: Receives signals from the interaction module and translates them into robot actions, managing the different system modes (selection, mapping, navigation).

Datasets

Not explicitly discussed in the paper. The system is evaluated using experiments in both simulated (Stage platform) and real indoor environments (corridors and offices).

Key Insights and Ideas

  • Bridging the Human-Robot Communication Gap: The core insight is that by integrating human voice and gestures with robot sensor data, we can create maps that are far more intuitive and accessible than purely geometric representations. This is crucial for making service robots practical for everyday users.
  • The Power of "Office 506": The paper demonstrates that semantic labels, even simple ones like room numbers, significantly enhance a robot's understanding of its environment and its ability to interact with humans.
  • Robust Localization is Key: The proposed localization-evaluate-relocalization method highlights the importance of robust self-localization for any navigation system, especially when dealing with dynamic environments or imperfect map data. The focus on modified particle weights shows a nuanced approach to improving localization accuracy.
  • Interactive Mapping is More Than Just Labeling: The system's ability to track users and respond to gestures adds a layer of interactivity that goes beyond simple voice commands, making the mapping process more dynamic and user-friendly.

Key Results

  • Successful Grid-Semantic Map Creation: The system successfully builds Grid-Semantic maps by combining visual tracking, voice labeling, and sensor data. Examples of corridor and office maps with semantic markers are shown.
  • Improved Localization: The proposed localization-evaluate-relocalization method shows promising results, with high success rates (e.g., 96.20% in office environments, 100% in some tests) and a reduced number of relocalization attempts needed. The modified maximum particle weight is identified as a key feature for successful localization.
  • Feasible Navigation: Experiments in both simulated and real environments demonstrate the feasibility of the system for navigation, with the robot able to plan and execute paths to designated semantic locations.

Limitations and Future Work

  • Limitations:
    • The paper notes that in simple environments like corridors, where laser scan data can be very similar across different positions, the particle swarm can sometimes converge to the wrong position.
  • Areas with missing map data (due to obstacles during mapping) can lead to localization failures if they are large enough to significantly affect particle weights.
  • The current semantic labeling is manual (via voice).
  • Future Work:
    • Extending the system from manual semantic labeling to automatic semantic labeling.
  • Developing more efficient ways to add richer semantic information to the map.

Summary updated

BibTeX
@article{zhao2015building,
  author = {Henry Cheng Zhao and Cheng Zhao and Weixing Mei and Wei Pan},
  title = {{Building a grid-semantic map for the navigation of service robots through human-robot interaction}},
  journal = {Digit. Commun. Networks},
  year = {2015},
  doi = {10.1016/j.dcan.2015.09.002},
}
Figure