Development of a Prototype AI System for Real-time Emotion Prediction and Mental State Adjustment

Akihiro Sasaki, Eriko Sugisaki, Roberto Legaspi, Yasushi Naruse, Nao KobayashiPERSUASIVE (Adjunct) 2024

Introduction

Emotional states significantly influence cognitive performance and daily well-being. However, most existing affective computing systems are limited by being "post-hoc"—analyzing data after an event—or by using overly simplistic binary classifications (e.g., positive vs. negative). This research presents a prototype AI system designed to bridge this gap by predicting complex emotional states in real-time using physiological signals, laying the groundwork for active mental state adjustment.

System Design and Methodology

The proposed system utilizes a multi-modal approach, integrating Electroencephalogram (EEG) and Electrocardiogram (ECG) data to monitor both central and autonomic nervous system activity.

Schematic diagram of the system design, illustrating the training and estimation phases.

The architecture relies on XGBoost, a gradient-boosted decision tree algorithm selected for its high computational efficiency. This choice allows the system to run on consumer-grade hardware while maintaining the fast training speeds necessary for real-time applications. The data pipeline involves wearable sensors transmitting data via Bluetooth to a smartphone bridge, which then relays the information to a central computer for processing.

To capture rapid emotional transitions, the system employs a 10-second sliding temporal window. Crucially, the model updates its prediction every 0.5 seconds, providing the low-latency feedback required for a closed-loop intervention system.

Data Collection and Emotional Modeling

The model was trained on a custom dataset comprising 2,322 instances from 54 participants. During the protocol, participants listened to music for one minute while their physiological data was recorded. Ground truth was established through self-reported ratings on a 15-point scale across six distinct emotional axes:

  • Sad–Happy
  • Nervous–Relaxed
  • Fear–Relieved
  • Lethargic–Excited
  • Depressed–Delighted
  • Angry–Serene

This multi-axis model allows the system to distinguish between physiologically similar but psychologically distinct states, such as "Nervous" and "Excited," both of which involve high arousal.

Results and Discussion

The prototype successfully demonstrates the feasibility of a real-time emotion-tracking pipeline. Preliminary benchmarks indicate that the system builds upon previous methodologies that achieved Mean Absolute Percentage Error (MAPE) rates of less than 25% for the Angry-Serene axis and 36% for the Fear-Relieved axis. By utilizing XGBoost, the system maintains a minimal resource footprint, suggesting high potential for future deployment on mobile devices.

The ultimate vision for this technology is "Mental State Adjustment." In this paradigm, the AI acts as a digital thermostat, subtly modifying the user's environment—such as generative music or lighting—to nudge them toward an optimal emotional state for their current task.

Limitations and Future Work

While the prediction pipeline is functional, several challenges remain. The researchers aim to improve accuracy, targeting a MAPE below 20% for both training and real-world deployment. Additionally, the "adjustment" component—the generative AI responsible for environmental changes—is still under development. Future iterations must also address the inherent "noise" in physiological data collected outside controlled laboratory environments, where movement and external distractions can degrade signal quality.

Summary updated

BibTeX
@inproceedings{sasaki2024development,
  author = {Akihiro Sasaki and Eriko Sugisaki and Roberto Legaspi and Yasushi Naruse and Nao Kobayashi},
  title = {{Development of a Prototype AI System for Real-time Emotion Prediction and Mental State Adjustment}},
  booktitle = {PERSUASIVE (Adjunct)},
  year = {2024},
}
Figure