Extracting Knowledge about Users' Activities from Raw Workstation Contents

Tom M. Mitchell, Sophie H. Wang, Yifen Huang, Adam CheyerAAAI 2006

Introduction

Intelligent personal agents require a deep understanding of a user's context to be truly effective. However, a significant "knowledge bottleneck" exists: most users do not manually organize their digital assets—emails, calendar events, and contacts—into logical projects or activities. This research introduces ActivityExtractor, a system designed to automatically discover and organize these latent activities from raw workstation data without requiring manual user intervention.

Methodology

ActivityExtractor employs a multi-stage process to bridge the gap between disparate data types:

  1. Cross-Modal Vectorization: Using Google Desktop Search (GDS) as a "Rosetta Stone," the system generates word distribution vectors for various entities. By querying GDS for a person's name or a meeting title, the system can represent these different modalities in a shared mathematical space.
  2. EM-Based Clustering: An Expectation-Maximization algorithm groups emails based on their textual content. The system utilizes "Guided Desktop Initialization" (GDI), which uses user-provided activity names to seed clusters more effectively than standard heuristics.
  3. Social Network Analysis (SNA): To refine clusters that may be topically similar but socially distinct, the system analyzes communication graphs. SNA helps split clusters where participants do not overlap, ensuring that distinct activities are not conflated simply because they share technical jargon.
  4. Frame Construction: The final output is an "activity frame"—a structured summary containing relevant meetings, contacts, and keywords.

Experimental Evaluation

The system was tested on real-world data from three distinct workstations: a university professor (Workstation A), a staff researcher (Workstation B), and a graduate student (Workstation C). These datasets varied in size and pre-existing organization, providing a robust test for the ActivityExtractor's versatility across different professional roles.

Results and Discussion

The evaluation demonstrated that ActivityExtractor is highly effective at identifying the core components of an activity. The system achieved high accuracy in identifying primary email senders and recipients across all tested workstations, often exceeding 90%.

Accuracy of field identification across workstations

A critical finding was the transformative impact of Social Network Analysis. While textual clustering provides a baseline, SNA significantly purifies the resulting activity frames. For instance, in Workstation B, accuracy improved from 0.66 to 0.83 following the SNA phase, and Workstation C saw a similar jump from 0.58 to 0.73.

Impact of Social Network Analysis (SNA) on overall accuracy

In specific test cases, such as a "CALO ActivityExtractor" cluster, users validated the system's accuracy at up to 98%, judging 240 out of 245 emails to be correctly categorized.

Conclusion

ActivityExtractor successfully automates the discovery of user activities by combining desktop search context with social signals. By leveraging existing search indices and social patterns, the system overcomes the knowledge bottleneck, providing a foundation for more capable and context-aware intelligent agents.

Summary updated

BibTeX
@inproceedings{mitchell2006extracting,
  author = {Tom M. Mitchell and Sophie H. Wang and Yifen Huang and Adam Cheyer},
  title = {{Extracting Knowledge about Users' Activities from Raw Workstation Contents}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2006},
}
Figure