Introduction
Navigating an unmapped indoor environment to locate a specific object—such as a "cup" or an "orange"—is a task humans perform intuitively but remains a significant challenge for robotics. Traditional approaches often rely on expensive LiDAR sensors for geometric mapping or require extensive training data for end-to-end reinforcement learning, which frequently fails to generalize to novel environments. Researchers at TCS Research have proposed a novel architecture that leverages semantic "common sense" and standard RGB cameras to bridge this gap, enabling robots to find targets efficiently in unknown spaces.
System Architecture and Methods
The proposed system centers on the GeoSem Map, a relational framework that integrates geometric data with semantic knowledge. The architecture consists of several interconnected modules designed to process visual information and make high-level navigation decisions.
Perception and Knowledge Representation
The perception pipeline utilizes a "Frame Filter" to select keyframes from the video stream. These frames are processed by YOLO v3 for fast object detection, with a fallback to Mask-RCNN for more complex scenes. Instead of treating the environment as a collection of pixels, the robot interprets it through semantic relationships stored in a knowledge base using OWL and RDF technologies. By training on the Visual Genome dataset, the robot understands probabilistic co-locations, such as a "cup" likely being found on a "table."
Decision Logic and the Landmark Score
To guide navigation, the system calculates a "Landmark Score," a heuristic that assesses the probability of a target's presence based on current visual cues and exploration history. If the score is low, the robot may backtrack to a previous high-probability zone. The decision module prioritizes moving through doors or toward "relational objects" (e.g., moving toward a table when searching for a cup) to narrow the search space.
Experimental Setup
The system was evaluated in both simulated and real-world environments.
The researchers created a custom indoor environment in Webots for a "gamified" comparison between the robot and human participants. Real-world testing was conducted using a Double2 wheeled robot in a standard office setting, where it successfully navigated to find targets like a "cup" hidden behind a "chair" by identifying intermediate cues like "bottles" and "tables."
Key Results and Insights
The "gamified" evaluation revealed that the robot significantly outperformed human users in a search-and-find task. While human participants (IT professionals) took an average of 632 seconds to find an "orange" in the simulation, the robot completed the same task in just 117 seconds.
This superior performance is attributed to:
- Semantic Reasoning: The robot uses a knowledge graph to follow a "semantic trail" (e.g., chair -> table -> cup) rather than wandering aimlessly.
- Occlusion Handling: The system explicitly models
occlusionByrelations, allowing the robot to infer that a target might be hidden and plan its path to gain a better vantage point. - Systematic Exploration: Unlike humans, who often struggled with spatial orientation in a 2D interface, the robot maintained a consistent and efficient exploration strategy.
Limitations and Future Work
Despite its success, the current model assumes a static environment and does not account for dynamic obstacles like moving people. Additionally, the reliance on pure RGB data can lead to challenges in precise obstacle avoidance compared to depth-sensing systems. Future work will focus on integrating "Stream Reasoning" for faster real-time decision-making and incorporating depth estimation to enhance navigation in complex, dynamic real-world settings.
Summary updated