TL;DR
This paper introduces HealthSHIELD, a low-cost system using Azure Kinect depth cameras to monitor workers' hand-to-face contact and social distancing in real-time. The system aims to improve hygiene compliance and reduce contagion risks in workplaces, achieving a high accuracy of 90.96% in distinguishing between high-risk and low-risk contact events.
What Problem Does This Paper Address?
The COVID-19 pandemic highlighted the critical need for effective hygiene practices in public spaces and workplaces to prevent the spread of infectious diseases. While social distancing and mask-wearing are important, hand-to-face contact remains a significant, often unconscious, transmission route. Existing solutions for monitoring these behaviors are either too expensive, intrusive, or lack the accuracy needed for real-time intervention. This paper addresses the challenge of developing an affordable, non-intrusive, and accurate system to monitor and encourage adherence to hygiene best practices, specifically hand-to-face contact avoidance and social distancing, in occupational settings.
Methods
The HealthSHIELD system utilizes the Microsoft Azure Kinect DK, a low-cost RGB-D sensor, for its depth sensing and body tracking capabilities. The system comprises four modules:
- Data Retrieval: Acquires 3D joint coordinates and infrared video from the Azure Kinect.
- Gesture Detection: Employs custom heuristics to detect hand-to-face contact events. It models the head as a sphere and defines a contact volume. If a thumb joint enters this volume, a contact is registered. The system also classifies the contact area on the face (mouth/nose, eyes, ears, or low-risk zones) based on the nearest joint to the thumb.
- Attitude Monitor: Compares consecutive frames to identify contact events and their areas, grouping them into sequences. It also monitors interpersonal distances between individuals.
- Graphical User Interface (GUI): Provides real-time feedback to users, displaying warnings for hygiene breaches and presenting statistical reports.
The system avoids machine learning for gesture detection, relying instead on geometric heuristics and predefined thresholds, making it computationally efficient and easier to implement.
Datasets
The paper describes a validation experiment using one volunteer. The ground truth for hand-face contact events and their classifications was meticulously annotated by a human experimenter who analyzed RGB recordings of the volunteer's gestures. This created a benchmark dataset for evaluating the HealthSHIELD system's performance.
Key Insights and Ideas
- Low-Cost Depth Sensing is Viable: The study demonstrates that affordable RGB-D sensors like the Azure Kinect can effectively track body movements and detect specific gestures relevant to hygiene.
- Heuristics Over ML for Simplicity: By using custom geometric heuristics rather than complex machine learning models for gesture detection, the system achieves good performance with lower computational overhead and easier implementation.
- Real-time Intervention is Key: The system's ability to provide immediate visual and acoustic warnings can significantly increase user awareness and encourage behavioral change.
- Distinguishing Risk Areas Matters: Classifying hand-to-face contacts into specific facial regions (e.g., mouth/nose vs. ears) allows for a more nuanced understanding of contagion risk.
Key Results
The preliminary validation study yielded promising results:
- Seven-Class Classification: The system achieved an overall accuracy of 79.83% in classifying hand-face contact events into six specific areas (mouth/nose, right eye, left eye, right ear, left ear, low risk) and occlusion events.
- Three-Class Classification: When simplified into "risk contacts" (mouth/nose, eyes), "low-risk contacts" (ears, low-risk area), and "occlusions," the system achieved a significantly higher accuracy of 90.96%.
- High Agreement with Human Observer: The system's classification showed substantial agreement with human observation, as indicated by Cohen's Kappa values of 0.735 for the seven-class and 0.876 for the three-class classification, suggesting "almost perfect agreement" in the latter.
- Real-time Performance: The system provided real-time alerts without noticeable delay.
Limitations and Future Work
The authors acknowledge several limitations:
- Occlusion Issues: The system's accuracy can be negatively affected by body part occlusions, particularly if the individual turns away from the sensor or if hands are close to the face, leading to potential segmentation errors.
- Limited Field of View: The system requires individuals to be within the sensor's field of view and to face it within a certain angle (less than 45 degrees turn).
- Single-Use Case Scenario: The preliminary testing was conducted in a single-use case scenario, and broader validation in diverse workplace environments is planned.
Future work includes:
- Extensive Validation: Conducting more comprehensive validation studies in various workplace settings.
- Training Tool Development: Evaluating the system's effectiveness as a training tool for hygiene best practices, leveraging real-time warnings and reports.
- User Acceptance Studies: Assessing user acceptance of the warning system and comparing it to other feedback mechanisms, such as wrist-worn vibrating bracelets.
- Data-Driven Insights: Utilizing the collected data to understand the importance of different contagion pathways and identify high-risk user categories for targeted interventions.
Summary updated