Introduction
Recognizing complex, goal-oriented human activities—such as "making tea" or "cleaning the bathroom"—remains a significant challenge in ubiquitous computing. While low-level physical actions like walking are easily detected via accelerometers, high-level activities require contextual understanding. Traditional supervised learning approaches are often bottlenecked by the need for massive, manually labeled datasets. This paper introduces SRCS (State Recognition using Common Sense), a system that bridges the gap between raw sensor data and high-level activity recognition by leveraging large-scale common-sense databases.
System Architecture
The SRCS system employs a multi-stage pipeline to transform discrete sensor events into probabilistic logical inferences. The process begins with a relational database of common-sense facts, which are then weighted, propositionalized into Horn clauses, and finally mapped onto a graphical model for temporal reasoning.
Methodology
Dense Sensing and Knowledge Integration
The system utilizes "dense sensing" via the iBracelet, a wearable RFID reader. When a user interacts with tagged household objects, the system logs the interaction. To interpret these interactions, SRCS integrates approximately 50,000 facts from the OMICS (Open Mind Indoor Common Sense) database. These facts provide the "common sense" necessary to link objects to activities (e.g., holding a toothbrush implies the activity "brushing teeth").
Web-Scale Validation and Probabilistic Reasoning
Because crowd-sourced databases like OMICS can be noisy, the researchers used KnowItAll to mine the web and assign weights to facts based on their frequency in online text. This "web-scale validation" filters out niche or incorrect information.
The logical structure of these facts is converted into a Markov Random Field (MRF). To account for the temporal nature of daily life, these MRFs are linked into a Chain Graph, allowing the system to maintain a persistent belief state that evolves over time. To ensure computational tractability, the system employs Query-Directed Pruning, focusing only on the local subgraph relevant to the current activities.
Experimental Results
The system was evaluated using 75 minutes of data from three users performing 15 distinct activities in a simulated home environment. Key findings include:
- High Accuracy: The system achieved an overall accuracy of 88.42% in tracking 24 different state variables.
- Impact of Web-Mining: Incorporating web-mined weights significantly improved precision, increasing it from 18.07% to 31.73% compared to unweighted models.
- Inference Efficiency: Pruning the graph to a depth of $d=2$ allowed for real-time inference while maintaining a recall of 53.07%.
Discussion and Future Work
While SRCS demonstrates the power of common-sense reasoning, several limitations remain. The current model is propositional and struggles to differentiate between multiple instances of the same object type. Furthermore, the "ground truth" for activities like "making tea" can be subjective. Future work will explore integrating additional sensor modalities, such as GPS and accelerometers, and automating the selection of relevant state variables to track.
Summary updated