Deep Reinforcement Learning Empowered Activity-Aware Dynamic Health Monitoring Systems

Ziqiang Ye, Yulan Gao, Yue Xiao, Zehui Xiong, Dusit Niyato

TL;DR

This paper introduces DActAHM, a novel smart healthcare monitoring system that uses Deep Reinforcement Learning (DRL) and the SlowFast model to dynamically adjust health metric monitoring based on user activities. By intelligently selecting which metrics to track, DActAHM significantly improves monitoring efficiency and reduces resource consumption compared to existing methods.

What Problem Does This Paper Address?

The proliferation of wearable devices in smart healthcare generates a massive amount of biosignal data. Current health monitoring systems often assume that medical devices track all relevant health metrics concurrently within their functional scope. While this ensures comprehensive data collection, it leads to significant issues:

  • Excessive Resource Use: Monitoring irrelevant health metrics consumes unnecessary computational power and energy, especially critical for battery-powered wearable devices.
  • Data Redundancy: Gathering extraneous data increases storage requirements and complicates analysis.
  • Inefficient Monitoring: A one-size-fits-all approach fails to adapt to varying user activities, leading to suboptimal monitoring and potential delays in critical interventions.

This paper aims to bridge this gap by proposing a system that intelligently adapts its monitoring strategy based on the user's current activity, thereby optimizing performance and cost-efficiency.

Methods

The proposed DActAHM system combines two key AI techniques:

  • SlowFast Model for Activity Recognition: This model is used to process raw video data from surveillance cameras to accurately identify and predict the user's current activity. The SlowFast architecture is known for its efficiency in capturing both slow and fast visual dynamics, making it suitable for recognizing a wide range of human actions.
  • Deep Reinforcement Learning (DRL) for Adaptive Monitoring: Once an activity is identified, a DRL framework is employed to dynamically decide which health metrics to monitor and with what priority. The authors specifically leverage the Deep Deterministic Policy Gradient (DDPG) algorithm, a popular choice for continuous control tasks. DRL allows the system to learn an optimal monitoring policy by interacting with the environment and receiving rewards based on the effectiveness and efficiency of its decisions.

The system operates in a loop: capture video -> recognize activity (SlowFast) -> determine monitoring strategy (DRL) -> monitor relevant metrics. The DRL agent learns to balance the "relevance" of a metric (how closely it's associated with the activity) and its "cost" (computational resources and energy required).

Datasets

  • Kinetics-400 dataset: Used for training and evaluating the SlowFast model's activity recognition capabilities. This dataset contains a large collection of human action videos, categorized into 30 distinct types relevant to the study (e.g., running, yoga, walking).

Key Insights and Ideas

  • Activity-Centric Monitoring is Key: The core insight is that health monitoring doesn't need to be constant and comprehensive. By understanding what a user is doing, the system can intelligently prioritize which health metrics are most important at that moment.
  • DRL as an Intelligent Scheduler: DRL acts as a sophisticated scheduler, learning to allocate limited computational and energy resources to monitor the most impactful health metrics for a given activity, rather than blindly collecting all data.
  • Balancing Performance and Cost: The DRL framework explicitly optimizes for a trade-off between monitoring performance (relevance of metrics) and cost (energy/computation), leading to a more efficient and effective system.
  • Visual Cues for Health Insights: Leveraging visual data from surveillance cameras to infer user activity provides a powerful, non-intrusive way to contextualize health monitoring.

Key Results

  • 27.3% Improvement: DActAHM demonstrated a significant improvement of 27.3% in gain compared to the best-performing baseline (a fixed monitoring strategy).
  • Outperforms Baselines: The system consistently outperformed three baseline strategies:
  • Classical: Monitors all metrics (lowest reward due to inefficiency).
  • Random: Arbitrarily monitors metrics (similar to Classical).
  • Fixed: Monitors a predefined set of metrics (better than Classical/Random but less adaptive).
  • Convergence: The DDPG algorithm used for DRL showed convergence, with rewards stabilizing after an initial learning phase, indicating the system learned an effective policy.

Limitations and Future Work

  • Reliance on Visual Data: The system's effectiveness is tied to the availability and quality of visual data for activity recognition. Privacy concerns related to widespread surveillance are also an implicit consideration.
  • Generalizability of Activity Set: While Kinetics-400 is extensive, the system's performance might vary with activities not well-represented in the dataset.
  • Future Work: The paper suggests that this adaptive approach has immense promise for optimizing resource allocation and improving the relevancy of data in digitally-enhanced urban health settings. Further research could explore more sophisticated activity recognition models, different DRL algorithms, and integration with a wider range of biosensors.

Summary updated

Abstract

In smart healthcare, health monitoring utilizes diverse tools and technologies to analyze patients' real-time biosignal data, enabling immediate actions and interventions. Existing monitoring approaches were designed on the premise that medical devices track several health metrics concurrently, tailored to their designated functional scope. This means that they report all relevant health values within that scope, which can result in excess resource use and the gathering of extraneous data due to monitoring irrelevant health metrics. In this context, we propose a Dynamic Activity-Aware Health Monitoring strategy (DActAHM), as a novel framework based on Deep Reinforcement Learning (DRL) and SlowFast Model, for striking a balance between optimal monitoring performance and cost efficiency while ensuring precise monitoring based on users' activities. Specifically, with the SlowFast Model, DActAHM efficiently identifies individual activities and captures these results for enhanced processing. Subsequently, DActAHM refines health metric monitoring in response to the identified activity by incorporating a DRL framework. Extensive experiments comparing DActAHM against three state-of-the-art approaches demonstrate it achieves 27.3% higher gain than the best-performing baseline that fixes monitoring actions over timeline.
BibTeX
@inproceedings{ye2024deep,
  author = {Ziqiang Ye and Yulan Gao and Yue Xiao and Zehui Xiong and Dusit Niyato},
  title = {{Deep Reinforcement Learning Empowered Activity-Aware Dynamic Health Monitoring Systems}},
  booktitle = {ICC},
  year = {2024},
  doi = {10.1109/icc51166.2024.10623033},
}

Loading related content...

Figure