Everyday Living Artificial Intelligence Hub

Raymond Finzel, Esha Singh, Martin Michalowski, Maria Gini, Serguei PakhomovNAACL 2021

TL;DR

This paper introduces the "Everyday Living AI Hub," a novel framework that integrates wearable sensors and conversational AI to provide personalized, real-time stress detection and intervention. By combining unobtrusive physiological monitoring with natural language interaction, the Hub aims to proactively manage stress and anxiety, particularly for vulnerable populations like elders aging in place.

What Problem Does This Paper Address?

The paper addresses the significant and growing problem of stress, which is a well-established contributor to numerous chronic diseases, including cardiovascular disease, type 2 diabetes, obesity, sleep disorders, depression, and even Alzheimer's disease. While the impact of stress is widely recognized, effectively detecting and intervening in real-time, especially in everyday living environments, remains a challenge. Existing approaches often rely on self-reported questionnaires or lack the integration of multiple sensing modalities and personalized interventions. The authors highlight the increasing prevalence of IoT devices in our homes and on our bodies, presenting an opportunity to develop holistic systems that can sense, integrate, and respond to a user's physical, emotional, and cognitive state in an accessible manner.

Methods

The Everyday Living AI Hub is a proof-of-concept framework that orchestrates three core components:

  1. Natural Language Interaction: A conversational voice assistant (Hub CA) powered by MindMeld is used for user interaction. Speech data is processed through Automatic Speech Recognition (ASR) and Text-To-Speech (TTS) services.
  2. Physiological Signal Sensing: Wearable devices and smart textiles are employed to collect data on activity, heart rate, body temperature, and skin electrical conductivity.
  3. On-Body Stimulation: Garment-based interventions, such as pressure and heat, are delivered to reduce detected stress and anxiety.

The framework utilizes industry-standard IoT protocols, including MQTT for lightweight publish/subscribe messaging, and a web service based on the Responder Python framework for managing user accounts and profiles. Data from wearable devices can be sent via Bluetooth to a companion phone app or directly to the Hub Server's AI collection point. The Hub CA can then trigger interventions via a web-service API, which can include notifications or direct control of "In Real Life (IRL)" devices, such as the SmartHugs Garment for compression.

Datasets

Not discussed in the paper.

Key Insights and Ideas

  • Holistic Stress Management: The paper proposes a shift from reactive stress management to a proactive, integrated approach that combines sensing, analysis, and intervention within the user's daily life.
  • Leveraging Ubiquitous Technology: The framework effectively leverages the increasing availability of IoT devices and conversational AI to create a personalized health and wellbeing system.
  • Human-in-the-Loop Design: The inclusion of a conversational agent allows for user confirmation of stress events and preferences for interventions, ensuring a more personalized and acceptable experience.
  • Smart Textiles for Intervention: The integration of smart textiles for delivering physical interventions like compression and heat offers a novel way to provide immediate, localized stress relief.

Key Results

The paper presents a proof-of-concept framework and discusses its architecture and components. Specific quantitative results regarding the accuracy of stress detection or the efficacy of interventions are not detailed in this initial work. The authors mention that preliminary studies have shown commercial IoT wearables can detect heart rate changes corresponding to stress, but machine learning models are still under development for real-time prediction. The Hub CA's dialog flows for stress management are also noted as being under development.

Limitations and Future Work

The authors acknowledge several challenges and areas for future work:

  • Real-time Stress Prediction: Existing machine-learning models are not yet equipped to reliably predict stressful events in real-time. Data collection and model development are ongoing.
  • Conversational Agent Development: While based on established chatbot paradigms, the dialog flows and models for the Hub CA are still under development for the specific domain of stress management.
  • Intervention Efficacy: Further research is needed to validate the effectiveness of the proposed interventions in reducing stress and anxiety.
  • Scalability and Deployment: The current work is a proof-of-concept, and future efforts will likely focus on scaling the system and deploying it in real-world settings.

Summary updated

BibTeX
@inproceedings{finzel2021everyday,
  author = {Raymond Finzel and Esha Singh and Martin Michalowski and Maria Gini and Serguei Pakhomov},
  title = {{Everyday Living Artificial Intelligence Hub}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2021},
}
Figure