Abstract
This paper introduces the Open Mind Indoor Common Sense (OMICS) project, a web-based platform designed to crowdsource domain-specific "common sense" knowledge from non-experts to assist indoor mobile robots. By collecting over 26,000 relations regarding household objects, human desires, and spatial layouts, the project demonstrates how robots can leverage this data to infer user intentions (e.g., desiring a cold drink) and identify room types based on observed objects. The study highlights the effectiveness of template-based acquisition and domain-restricted knowledge bases in making robots more intuitive in human environments.
Introduction
While mobile robots have advanced significantly in navigation and sensing, they frequently lack the "obvious" knowledge humans take for granted. For instance, a robot may perceive a warm soda on a table but fail to understand that humans generally prefer soda to be cold, or that a refrigerator is the appropriate location to address this state. Traditional AI efforts to codify human knowledge, such as the Cyc project, are often too broad or labor-intensive for specific robotic applications. The authors argue that for robots to be effective in homes and offices, they require dense, domain-specific knowledge gathered efficiently from the general public.
Methods and Technical Approach
The researchers developed the OMICS website, which utilizes sentence templates to prompt users for information. This structured approach bypasses complex Natural Language Processing (NLP) requirements by implicitly defining relationships through fill-in-the-blank prompts.
Key Architectural Components:
- Object-Centric Representation: Knowledge is stored as $(object, property)$ pairs, simplifying complex logic into manageable relations like
(cup-of-coffee, hot). - Dynamic Prompting: The system iteratively generates new templates based on previous user input. If a user introduces a new object like a "microwave," the system subsequently asks others about its location or function.
- Bayesian Inference for Room Labeling: To identify rooms, the robot employs a generative Bayesian model. It combines real-time sensor data (e.g., detecting a sink) with probabilities from the OMICS database to determine the most likely room label (e.g., kitchen vs. bedroom).
- BDI Framework: The system uses a Belief-Desire-Intention architecture. The robot observes a state (Belief), uses common sense to infer human needs (Desire), and selects an action to achieve the goal (Intention).
Datasets
- OMICS Dataset: A corpus of approximately 29,000 submissions from over 400 users, resulting in ~26,000 high-quality relations after filtering.
- WordNet Integration: Used for word sense disambiguation and knowledge generalization (e.g., categorizing "apples" and "oranges" as "fruit").
- Object Image Collection: A set of 400+ indoor object images used to train object recognition systems through user-led labeling.
Key Insights
- Domain Restriction: Limiting scope to "indoor home and office" environments resulted in higher data density and immediate actionability compared to general-purpose common sense projects.
- Templates as Parsers: Using templates ensures data is "machine-ready" upon submission, avoiding the pitfalls of free-form text analysis.
- Causality Challenges: Users found the "Causes" activity (e.g., "X causes Y") significantly more difficult than spatial or functional tasks, leading to higher rejection rates for causal data.
Results and Practical Reasoning
The project successfully gathered 18,000 submissions within months, growing to over 29,000 by early 2004. Despite the open nature of the platform, only 10% of entries required rejection. The authors demonstrated a successful logic chain for "Active Desire Selection," where a robot observing a warm beverage can infer the need for a chilled one and identify the necessary causal action.
Limitations and Future Work
Currently, the system requires manual administrative review, taking roughly 30 seconds per page to vet entries. Furthermore, while the robot can determine what should be done (e.g., "the coffee should be hot"), it lacks the mid-level physical motor skills to execute these tasks autonomously. Future research will focus on automated quality control and bridging the gap between high-level reasoning and physical execution in noisy, real-world environments.
Summary updated