TL;DR
This paper introduces SONAR, a novel dataset of nursing activities captured using inertial sensors in a real-world nursing home setting. It addresses the critical need for accurate nursing documentation and improved patient care by providing a rich dataset and benchmark models for human activity recognition in this complex domain.
What Problem Does This Paper Address?
Accurate and comprehensive documentation is vital for quality patient care, but it often contributes to a significant nursing workload. Human Activity Recognition (HAR) using wearable sensors offers a potential solution by automating the recording of nursing tasks. However, existing HAR datasets often focus on simpler activities of daily living (ADL) or are collected under controlled laboratory conditions, failing to capture the nuances of real-world nursing environments. This gap limits the development of robust HAR systems for healthcare. The SONAR dataset aims to bridge this gap by providing a comprehensive collection of nursing activities performed by professional nurses in an actual elderly care facility.
Methods
The researchers collected data using five Xsens DOT v2 inertial measurement units (IMUs) worn by 14 professional nurses. These sensors captured acceleration, angular velocity, and orientation data. To ensure accurate labeling, a custom application was developed that synchronized sensor data with real-time pose estimation from parallel video streams. This allowed for precise annotation of 23 distinct nursing activities. The study also involved training and evaluating three deep learning models: CNN-LSTM, ResNet, and DeepConvLSTM, to benchmark the dataset's utility for activity recognition. Various cross-validation techniques (k-fold, leave-recordings-out, leave-one-subject-out) and performance metrics (accuracy and F1-score) were employed to assess model performance across different sensor combinations.
Datasets
The primary dataset introduced in this paper is SONAR (Sensor-Oriented Nursing Activity Recognition). This dataset comprises 61.7 hours of data from 14 nurses performing 23 distinct nursing activities in a real-world nursing home. The data includes 14 sensor streams (acceleration and angular velocity) from five IMUs. The paper also references several other HAR datasets for comparison, including OPPORTUNITY, PAMAP2, Skoda, UCI HAR, SHL, and HARTH.
Key Insights and Ideas
- Real-world complexity matters: The SONAR dataset highlights that activities in a real nursing home are far more complex and varied than those typically found in laboratory settings or datasets focused on ADLs. This complexity is crucial for developing truly applicable HAR systems.
- Sensor fusion is key: The study demonstrates that combining data from multiple sensors, particularly wrist and pelvis sensors, significantly improves activity recognition performance. This suggests that a multi-sensor approach is essential for capturing the subtle movements involved in nursing tasks.
- Deep learning models show promise: The benchmark results indicate that deep learning models, especially CNN-LSTM, can achieve reasonable performance in recognizing nursing activities from inertial sensor data, even with the inherent challenges of real-world data.
- Pose estimation aids labeling: The innovative use of pose estimation for data labeling proved effective in ensuring accurate and privacy-preserving annotation, a critical aspect for sensitive healthcare data.
Key Results
The SONAR dataset contains 61.7 hours of data, encompassing 23 nursing activities performed by 14 nurses. The benchmark evaluation showed that the CNN-LSTM model achieved the best performance across various sensor combinations and cross-validation strategies. For instance, with a combination of wrist, pelvis, and ankle sensors ({LW, RW, ST, LF, RF}), the CNN-LSTM model achieved an accuracy of 0.44 (±0.06) and an F1-score of 0.4 (±0.06) using k-fold cross-validation. The study also found that wrist sensors (LW and RW) were particularly crucial for performance, with the addition of the pelvis sensor (ST) further enhancing results.
Limitations and Future Work
- Dataset size: While 61.7 hours is substantial, the authors acknowledge that larger datasets would further improve model generalization.
- Activity granularity: The current dataset categorizes activities into 23 distinct types. Future work could explore finer-grained activity recognition.
- Generalizability: While collected in a real-world setting, the dataset represents one specific nursing home. Further validation across different facilities and patient populations would be beneficial.
- Model complexity: The paper focuses on three deep learning models. Exploring other architectures, including transformer-based models, could yield further improvements.
- Privacy considerations: While pose estimation aids anonymization, the ethical implications of continuous monitoring in healthcare settings warrant ongoing consideration.
The authors suggest that the SONAR dataset can be used to develop more sophisticated HAR systems, potentially leading to improved nursing documentation, reduced workload, and ultimately, enhanced patient care. The availability of the dataset and associated code on Zenodo and GitHub facilitates further research and collaboration in this important area.
Summary updated