Abstract
Autonomous service robots operating in home environments must understand the semantic layout of their surroundings to execute human commands effectively. This paper introduces SpCoMapping, a novel statistical framework that enables robots to learn the names and physical boundaries of rooms through unsupervised, multimodal learning. By combining spatial concept acquisition with Markov Random Fields (MRFs), the system allows robots to discover complex, arbitrary room shapes using raw sensor data and natural user utterances, bypassing the need for predefined place names.
Introduction
For a service robot to follow a command like "clean Joseph's room," it must solve two fundamental challenges: grounding (linking the linguistic term to a specific location) and segmentation (defining the physical extent of that location). Traditional semantic mapping methods often struggle with the "overwrite problem," where new observations erase previous statistical data, or they rely on rigid geometric assumptions (like ellipses) that fail to capture the actual layout of human homes. Furthermore, many systems use closed vocabularies, preventing them from adapting to user-specific terminology.
Methods
SpCoMapping addresses these issues by integrating probabilistic spatial concept acquisition with a Markov Random Field (MRF). The framework fuses three distinct data streams: visual data from a CNN, spatial data from self-localization, and linguistic data from user utterances.
Unlike previous models that used Gaussian distributions to represent areas, the MRF models dependencies between neighboring pixels on an occupancy grid. This allows the robot to learn arbitrary shapes that respect physical boundaries like walls. Using a Dirichlet Process (DP) prior, the robot can automatically estimate the number of distinct categories (rooms) without prior knowledge. Inference is performed using Gibbs sampling to iteratively refine the semantic map, word distributions, and visual features.
Experimental Setup and Datasets
The system was evaluated in both simulated and real-world environments. Simulation experiments utilized SIGVerse, testing across five distinct house layouts. Real-world data was collected using a Toyota Human Support Robot (HSR) equipped with an RGB-D camera and laser range sensor in a laboratory living space. The CNN component utilized the Places205 dataset for initial scene recognition capabilities.
Key Results
The researchers compared SpCoMapping against several baselines using the Adjusted Rand Index (ARI) to measure clustering accuracy.
- Clustering Performance: SpCoMapping (DP) achieved an average ARI of 0.5066 in simulation, significantly outperforming a CNN-based baseline (0.2795) and previous Gaussian-based spatial concept methods (0.3499).
- Qualitative Success: In real-world deployments, the robot successfully generated semantic maps where regions like "kitchen" and "meeting space" were correctly bounded by the room's physical geometry.
- Word Grounding: The system demonstrated high accuracy in locating rooms when prompted with specific words by calculating the highest probability region associated with those vocabulary features.
Discussion and Future Work
SpCoMapping represents a step toward "symbol emergence," where robots ground language in the physical world through consistent co-occurrence of visual patterns, coordinates, and user utterances. By treating walls as hard constraints within the MRF, the model avoids the "bleeding" effect common in other semantic mapping techniques.
However, several limitations remain. The current implementation is a batch process; future work aims to develop an online learning version integrated with SLAM for real-time adaptation. Additionally, extending the model from 2D to 3D semantic space will be crucial for complex object manipulation tasks. Finally, improving natural language processing to filter out linguistic noise will enhance the robustness of word grounding.
Summary updated