Abstract
Community-Guided Learning (CGL) addresses the challenges of crowd-sourced sensor data, characterized by inconsistent and overlapping user labels. By automatically refining these labels through a process of splitting broad categories into sub-classes and merging semantically similar labels based on classifier performance, CGL develops robust activity models. Our evaluation demonstrates that CGL significantly outperforms traditional supervised methods, particularly in scenarios with significant labeling noise or "pollution."
Introduction
The development of "people-aware" mobile applications relies heavily on large-scale, labeled sensor data. However, obtaining precise labels from casual users is difficult due to two main issues: semantic inconsistency (different labels for the same activity or one label for diverse activities) and boundary errors (imprecise start/stop times for labeling sessions). Traditional supervised learning often fails under these conditions as it assumes rigid, perfect labels. Community-Guided Learning (CGL) offers a middle ground, treating user labels as "soft" hints and allowing the underlying sensor data structure to refine classification categories.
The CGL Framework
CGL introduces a multi-stage pipeline designed to refine user-provided labels before and during classifier training.
Intra-class Splitting
The system analyzes data within a single label (e.g., "Driving") using Euclidean k-means clustering. By identifying the "elbow" in the objective function, CGL determines if a label should be split into distinct sub-classes, such as urban driving versus stationary idling. This captures the physical diversity often hidden under a single semantic tag.
Inter-class Merging
After initial training, CGL evaluates the similarity between sub-classes based on cross-classification performance (combined f-score). If a classifier for one class accurately predicts another, the classes are merged into a unified category, resolving semantic inconsistencies where different users use different terms for the same behavior.
Experimental Methodology
The framework was evaluated using a 33-hour audio dataset collected by five researchers using mobile devices. Users provided freeform labels like "working," "eating," and "gym." To capture the acoustic signature of environments, 39 features were extracted across time (ZCR, RMS), frequency (spectral entropy, bandwidth), and cepstral (MFCCs) domains. The researchers also introduced "synthetic pollution" by splicing data from different activities to simulate real-world boundary errors.
Results
CGL demonstrated superior performance across various scenarios compared to naive supervised learning, as shown in the table below.
Key findings include:
- Splitting Success: Identifying three distinct clusters within the "Driving" class improved recall from 73.0% to 76.5%.
- Merging Success: Merging semantically related categories like "eating" and "communal places" improved the f-score from approximately 88% to 91%.
- Robustness to Pollution: In the "gym/driving" scenario, where data was heavily polluted, CGL achieved 95.5% accuracy, a 41.5% improvement over the naive baseline (69.5%). By splitting the pollution into its own sub-class, CGL effectively ignored the noise.
Conclusion
Community-Guided Learning effectively leverages noisy crowd-sourced labels to build high-performance activity classifiers. By allowing data distributions to guide label refinement, the system overcomes common human labeling errors. Future work will focus on improving scalability for real-time mobile execution and exploring online active learning to further minimize user labeling effort.
Summary updated