A Demonstration of the RADAR Personal Assistant

Andrew Faulring, Brad A. Myers, Ken Mohnkern, Michael FreedAAAI 2008

Abstract

RADAR (Reflective Agents with Distributed Adaptive Reasoning) is an AI-driven personal assistant designed to bridge the gap between email and task management. By employing machine learning to extract tasks from messages and prioritize workloads, RADAR significantly boosts user productivity. Evaluations demonstrate a 32% performance improvement in complex, multi-tasking scenarios compared to non-learning baselines, with the system's effectiveness increasing as the underlying models mature.

Introduction

Most professionals utilize their email inbox as a de facto to-do list, despite email clients lacking native task management features. This message-centric workflow often results in email overload, where critical information is obscured and deadlines are missed. While previous research has explored hybrid email/task applications, the RADAR project proposes a distinct approach: maintaining separate email and task functions while coordinating them through a personalized software agent. The primary challenge lies in developing an agent capable of "learning in the wild"—improving its ability to extract parameters, prioritize schedules, and automate data entry through passive observation of user behavior.

System Architecture

RADAR is composed of three primary subsystems utilizing a combination of standard and novel AI techniques:

  1. METAL (MEssage-TAsk Linking): This layer manages the transition from text to action. It employs regularized logistic regression to classify emails into eight task types. An NLP pipeline, utilizing the MinorThird system, extracts specific task parameters such as dates, names, and locations.
  2. MCA (Multi-task Coordination Assistance): Serving as the system's cognitive core, the MCA includes a Task Prioritizer that learns optimal operation sequences by observing experts. It also features an Attention Manager that uses models of user interruptibility to determine when to alert users about high-priority tasks.
  3. Powertools: These specialized applications facilitate task execution. A key example is the SpaceTime Planner (STP), which uses a novel optimization algorithm for mixed-initiative allocation under uncertainty, assisting users in scheduling complex events while suggesting constraint relaxations when necessary.

Experimental Methodology

The system was evaluated using a high-fidelity simulation rather than static datasets. Participants were tasked with organizing a fictional academic conference, inheriting an inbox of 119 emails. The evaluation compared a baseline version (RADAR -L) against a version with learning enabled (RADAR +L). The learning-enabled version was trained via simulated use and observations of expert behavior prior to participant interaction.

Results and Analysis

Annual evaluations conducted as part of DARPA’s PAL program revealed significant performance gains. In the third year of the study, the learning-enabled system (RADAR +L) achieved a performance score of 0.705, compared to 0.534 for the baseline (RADAR -L), representing a 32% improvement.

Performance scores for RADAR with (+L) and without (-L) learning across two years of evaluation.

The data indicates that the "contribution of learning"—the performance gap between the untrained and trained systems—increased by 41% between Year 2 and Year 3. This suggests that the underlying machine learning components became significantly more effective as the project matured. Furthermore, users were able to process 119 emails and execute complex scheduling tasks within a strict two-hour window, a level of efficiency difficult to achieve with standard email tools.

Discussion and Future Work

A key insight from the project is the importance of mixed-initiative design. RADAR provides a "to-do style" interface where users can verify and correct agent extractions, creating a feedback loop that improves ML models over time. Additionally, usability testing led to a shift in terminology from "Task List" to "Action List," which users found more intuitive.

However, several limitations remain. The Task Shedding Advisor, designed to identify tasks for removal during high-pressure periods, was not fully integrated into the final results. Furthermore, the system relies on user verification; if extraction error rates are too high, the cognitive load of correction may offset productivity gains. Future research will focus on applying RADAR to the unpredictable nature of real-world corporate and personal email environments beyond controlled simulations.

Summary updated

BibTeX
@inproceedings{faulring2008demonstration,
  author = {Andrew Faulring and Brad A. Myers and Ken Mohnkern and Michael Freed},
  title = {{A Demonstration of the RADAR Personal Assistant}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2008},
}
Figure