Community-Guided Learning: Exploiting Mobile Sensor Users to Model Human Behavior

Daniel Peebles, Hong Lu, Nicholas D. Lane, Tanzeem Choudhury, Andrew T. CampbellAAAI 2010 · DOI

Abstract

Community-Guided Learning (CGL) addresses the challenges of crowd-sourced sensor data, characterized by inconsistent and overlapping user labels. By automatically refining these labels through a process of splitting broad categories into sub-classes and merging semantically similar labels based on classifier performance, CGL develops robust activity models. Our evaluation demonstrates that CGL significantly outperforms traditional supervised methods, particularly in scenarios with significant labeling noise or "pollution."

Introduction

The development of "people-aware" mobile applications relies heavily on large-scale, labeled sensor data. However, obtaining precise labels from casual users is difficult due to two main issues: semantic inconsistency (different labels for the same activity or one label for diverse activities) and boundary errors (imprecise start/stop times for labeling sessions). Traditional supervised learning often fails under these conditions as it assumes rigid, perfect labels. Community-Guided Learning (CGL) offers a middle ground, treating user labels as "soft" hints and allowing the underlying sensor data structure to refine classification categories.

The CGL Framework

CGL introduces a multi-stage pipeline designed to refine user-provided labels before and during classifier training.

The CGL pipeline: from community-sourced imperfect labels to refined sub-classes and final merged categories.

Intra-class Splitting

The system analyzes data within a single label (e.g., "Driving") using Euclidean k-means clustering. By identifying the "elbow" in the objective function, CGL determines if a label should be split into distinct sub-classes, such as urban driving versus stationary idling. This captures the physical diversity often hidden under a single semantic tag.

Inter-class Merging

After initial training, CGL evaluates the similarity between sub-classes based on cross-classification performance (combined f-score). If a classifier for one class accurately predicts another, the classes are merged into a unified category, resolving semantic inconsistencies where different users use different terms for the same behavior.

Experimental Methodology

The framework was evaluated using a 33-hour audio dataset collected by five researchers using mobile devices. Users provided freeform labels like "working," "eating," and "gym." To capture the acoustic signature of environments, 39 features were extracted across time (ZCR, RMS), frequency (spectral entropy, bandwidth), and cepstral (MFCCs) domains. The researchers also introduced "synthetic pollution" by splicing data from different activities to simulate real-world boundary errors.

Results

CGL demonstrated superior performance across various scenarios compared to naive supervised learning, as shown in the table below.

Performance comparison between naive supervised learning and CGL across different activity scenarios.

Key findings include:

  • Splitting Success: Identifying three distinct clusters within the "Driving" class improved recall from 73.0% to 76.5%.
  • Merging Success: Merging semantically related categories like "eating" and "communal places" improved the f-score from approximately 88% to 91%.
  • Robustness to Pollution: In the "gym/driving" scenario, where data was heavily polluted, CGL achieved 95.5% accuracy, a 41.5% improvement over the naive baseline (69.5%). By splitting the pollution into its own sub-class, CGL effectively ignored the noise.

Conclusion

Community-Guided Learning effectively leverages noisy crowd-sourced labels to build high-performance activity classifiers. By allowing data distributions to guide label refinement, the system overcomes common human labeling errors. Future work will focus on improving scalability for real-time mobile execution and exploring online active learning to further minimize user labeling effort.

Summary updated

BibTeX
@inproceedings{peebles2010community,
  author = {Daniel Peebles and Hong Lu and Nicholas D. Lane and Tanzeem Choudhury and Andrew T. Campbell},
  title = {{Community-Guided Learning: Exploiting Mobile Sensor Users to Model Human Behavior}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2010},
  doi = {10.1609/aaai.v24i1.7731},
}

Loading related content...

Figure