Introduction
Intelligent personal agents require a deep understanding of a user's context to be truly effective. However, a significant "knowledge bottleneck" exists: most users do not manually organize their digital assets—emails, calendar events, and contacts—into logical projects or activities. This research introduces ActivityExtractor, a system designed to automatically discover and organize these latent activities from raw workstation data without requiring manual user intervention.
Methodology
ActivityExtractor employs a multi-stage process to bridge the gap between disparate data types:
- Cross-Modal Vectorization: Using Google Desktop Search (GDS) as a "Rosetta Stone," the system generates word distribution vectors for various entities. By querying GDS for a person's name or a meeting title, the system can represent these different modalities in a shared mathematical space.
- EM-Based Clustering: An Expectation-Maximization algorithm groups emails based on their textual content. The system utilizes "Guided Desktop Initialization" (GDI), which uses user-provided activity names to seed clusters more effectively than standard heuristics.
- Social Network Analysis (SNA): To refine clusters that may be topically similar but socially distinct, the system analyzes communication graphs. SNA helps split clusters where participants do not overlap, ensuring that distinct activities are not conflated simply because they share technical jargon.
- Frame Construction: The final output is an "activity frame"—a structured summary containing relevant meetings, contacts, and keywords.
Experimental Evaluation
The system was tested on real-world data from three distinct workstations: a university professor (Workstation A), a staff researcher (Workstation B), and a graduate student (Workstation C). These datasets varied in size and pre-existing organization, providing a robust test for the ActivityExtractor's versatility across different professional roles.
Results and Discussion
The evaluation demonstrated that ActivityExtractor is highly effective at identifying the core components of an activity. The system achieved high accuracy in identifying primary email senders and recipients across all tested workstations, often exceeding 90%.
A critical finding was the transformative impact of Social Network Analysis. While textual clustering provides a baseline, SNA significantly purifies the resulting activity frames. For instance, in Workstation B, accuracy improved from 0.66 to 0.83 following the SNA phase, and Workstation C saw a similar jump from 0.58 to 0.73.
In specific test cases, such as a "CALO ActivityExtractor" cluster, users validated the system's accuracy at up to 98%, judging 240 out of 245 emails to be correctly categorized.
Conclusion
ActivityExtractor successfully automates the discovery of user activities by combining desktop search context with social signals. By leveraging existing search indices and social patterns, the system overcomes the knowledge bottleneck, providing a foundation for more capable and context-aware intelligent agents.
Summary updated