Predicting the Quality of User Experiences to Improve Productivity and Wellness

Priya Lekha Donti, Jacob Rosenbloom, Alex Gruver, James C. Boerkoel Jr.AAAI 2015 · DOI

Abstract

Modern productivity tools often fail because they place a high cognitive burden on users to manually input schedules and preferences. This paper introduces PaWPal, a smartphone-based "productivity pal" that leverages adaptive Experience Sampling (ESM) and Support Vector Regression (SVR) to predict "flow" states—periods of deep immersion and high effectiveness. By analyzing calendar data and user feedback, PaWPal generates a personalized heat map of predicted performance, helping users balance demanding workloads with personal wellness.

Introduction

College students frequently struggle to maintain a balance between academic rigor and personal well-being. A common pitfall is attempting to force productivity during periods of low focus, leading to burnout and inefficiency. While automated scheduling agents aim to solve this, they often require complex manual inputs that users find intrusive or burdensome.

The researchers behind PaWPal propose a shift in responsibility: the system should learn to predict a user's productivity rather than requiring the user to define it. By identifying "flow"—a psychological state characterized by total immersion in an activity—the system can provide actionable insights into peak performance windows. The primary challenge is collecting sufficient data for these predictions without causing "survey fatigue."

Methodology: The PaWPal System

The PaWPal application for Android integrates three core components to monitor and predict user experience:

1. Adaptive Experience Sampling (ESM)

Traditional ESM pings users at random intervals, which can be disruptive. PaWPal employs an adaptive approach by syncing with the user’s Google Calendar. It intelligently schedules notifications during free blocks and scales back during busy periods (e.g., classes or meetings) to minimize interruption.

2. Predictive Modeling with SVR

The system utilizes Support Vector Regression (SVR) to forecast user efficacy across four domains: curricular, extracurricular, social, and restorative. Three distinct regression models are trained to predict:

  • Immersion (Imm): The depth of engagement in a task.
  • Effectiveness (Eff): The quality and speed of work.
  • Emotional State (Em): The user's current mood.

3. Heat Map Visualization

To make these predictions accessible, PaWPal features a home screen heat map. Unscheduled time blocks are shaded with varying saturation; darker regions indicate times when the user is predicted to be most effective for a specific activity category.

Heat map visualization of predicted user effectiveness across different activity categories.

Data Collection and Pilot Study

A week-long pilot study was conducted in April 2014 at Harvey Mudd College with 25 student participants. Data was collected using the PACO (Personal Analytics Companion) app, with participants surveyed eight times daily regarding their context and experience. The resulting dataset was hand-tagged with temporal features (time of day, day of week) and contextual data (calendar status).

Results and Performance

The SVR model's predictive accuracy was evaluated against two naïve baselines: a Mode Classifier (predicting the most frequent response) and a Mean Classifier (predicting the average response). Performance was measured using Mean Absolute Error (MAE), where lower values indicate higher accuracy.

Comparison of SVR model performance against baseline classifiers using Mean Absolute Error (MAE).

As shown in the results, the SVR model significantly outperformed both baselines across all three metrics. Specifically, the model achieved an MAE of 0.63 for Immersion, 0.57 for Effectiveness, and 0.53 for Emotional State. These results demonstrate that even a limited set of temporal and calendar features can effectively capture individual productivity patterns.

Discussion and Key Insights

The study yielded several qualitative insights:

  • Increased Self-Awareness: Participants reported that the ESM surveys themselves were beneficial, as the process of reflection made them more aware of their own habits and energy levels.
  • Contextual Correlation: High-level features such as the day of the week and social context (interacting with others) were found to be strong predictors of flow states.
  • Informing vs. Directing: Rather than acting as a rigid scheduler, PaWPal functions as a mirror, providing users with the data needed to make informed decisions about their time management.

Limitations and Future Directions

While promising, the current model has limitations. The initial evaluation predicted efficacy across the entire group; future iterations will focus on individualized models to better account for personal variations like "night owl" or "early bird" tendencies. Additionally, the researchers plan to incorporate non-temporal features, such as GPS location and social interaction data, to further refine accuracy. The system also relies heavily on the accuracy of the user's digital calendar, which remains a potential point of failure if events are not logged.

A more extensive long-term study is planned to evaluate the app's sustained impact on student wellness and productivity over time.

Summary updated

BibTeX
@inproceedings{donti2015predicting,
  author = {Priya Lekha Donti and Jacob Rosenbloom and Alex Gruver and James C. Boerkoel Jr.},
  title = {{Predicting the Quality of User Experiences to Improve Productivity and Wellness}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2015},
  doi = {10.1609/aaai.v29i1.9740},
}
Figure