Does Elderly Enjoy Playing Bingo with a Robot? A Case Study with the Humanoid Robot Nadine

Nidhi Mishra, Gauri Tulsulkar, Hanhui Li, Nadia Magnenat-Thalmann, Lim Hwee Er, Lee Mei Ping, Cheng Siok KhoongCGI 2021 · OpenReview

TL;DR

This paper explores the impact of a humanoid social robot, Nadine, on the engagement and well-being of elderly individuals during a Bingo game in a nursing home. By employing computer vision techniques to analyze participant emotions and movements, the study found that Nadine's presence led to increased smiling, reduced overall movement, and a decrease in staff workload, suggesting a positive influence on both residents and caregivers.

What Problem Does This Paper Address?

The global population is aging, leading to an increased demand for elder care services. However, the workforce available to meet these needs is not growing at the same pace, creating a strain on healthcare systems and caregivers. This shortage impacts the quality of care, particularly in addressing the psychosocial well-being of the elderly. The paper proposes using social robots, specifically the humanoid robot Nadine, to host recreational activities like Bingo. This aims to alleviate caregiver workload, provide consistent engagement for residents, and assess whether such robots can foster comfort and enjoyment among the elderly, thereby improving their overall quality of life.

Methods

The study employed a mixed-methods approach, combining qualitative observations with quantitative analysis using computer vision.

  • Human-Robot Interaction Study: Nadine, a socially intelligent humanoid robot with realistic appearance and expressive capabilities, was deployed in a nursing home to host Bingo sessions. These sessions were compared to traditional Bingo sessions hosted by human care staff.
  • Computer Vision Analysis: Video recordings of the Bingo sessions were analyzed using several Deep Neural Networks (DNNs):
  • Face Detector: To identify and locate elderly participants.
  • Expression Recognizer (CNN with ResNet-50): To detect emotions, specifically distinguishing between "smiling" and "neutral" expressions.
  • Action Detector (Temporal Segment Network): To measure the physical movement of the elderly.
  • Optical Flow Estimator: To quantify the overall movement of care staff within the environment.
  • Quantitative Metrics: Three primary metrics were derived:
  • Happiness: Based on the probability of smiling expressions.
  • Movement (Elderly): Based on the confidence of detected actions.
  • Activity (Care Staff): Based on the average magnitude of optical flow.
  • Statistical Analysis: Independent samples t-tests were used to compare the metrics between Nadine-hosted and care-staff-hosted sessions. Bivariate correlations were also performed to assess changes over time.

Datasets

The study did not explicitly mention training datasets for the DNNs, but it did reference existing libraries and datasets for their implementation:

  • Dlib library: Used for the face detector.
  • CelebA dataset: Used for training the expression recognizer.
  • MMAction2 library: Provided the pre-trained temporal segment network for the action detector.
  • Real-world data: 24 Bingo sessions hosted by Nadine and 2 sessions hosted by care staff were recorded and analyzed.

Key Insights and Ideas

  • Humanoid Appearance Matters: The paper emphasizes that Nadine's realistic humanoid appearance and ability to emote are crucial for bridging the technology gap with the elderly, making interactions more natural and beneficial.
  • Early Positive Impact: The study suggests that the positive effects of Nadine's presence can be observed very early, with no significant period of adaptation required for the elderly to experience benefits.
  • Reduced Caregiver Burden: A significant finding was the lower optical flow (indicating less staff movement) during Nadine-hosted sessions, implying a reduction in the physical workload for care staff.
  • Calmer, Happier Residents: The increased "smile" metric and decreased "body score" (elderly movement) suggest that residents were more engaged and calmer when interacting with Nadine.

Key Results

  • Smile: The "smile" variable was significantly higher in sessions hosted by Nadine (mean 0.019) compared to care-staff-hosted sessions (mean 0.010).
  • Body Score (Elderly Movement): Elderly participants showed significantly less movement when Nadine was present (mean 0.154) compared to when care staff hosted (mean 0.172).
  • Optical Flow (Care Staff Movement): The optical flow was significantly lower in Nadine-hosted sessions (mean 0.181) compared to care-staff-hosted sessions (mean 0.288), indicating reduced staff activity.
  • Neutral Expression: No significant difference was found in the "neutral" expression metric between the two conditions.
  • No Significant Change Over Time: Bivariate correlations showed no significant linear change in the measured variables over the course of the sessions, suggesting immediate and sustained positive effects.

Limitations and Future Work

  • Limited Sample Size for Comparison: While 24 sessions with Nadine were conducted, only 2 sessions with care staff were used as a baseline, which could limit the generalizability of the comparative findings.
  • Focus on Specific Activity: The study focused solely on Bingo. Future work should explore Nadine's effectiveness in hosting other recreational activities.
  • Generalizability of Findings: The study was conducted in one nursing home. Further research is needed to determine the settings and conditions under which humanoid robots can be most beneficial.
  • Long-Term Impact: While immediate effects were observed, the long-term impact of Nadine's presence on the elderly's well-being and social interaction warrants further investigation.
  • Adaptation and Customization: The paper mentions adapting Nadine's modules and configurations for the nursing home environment. Future work could explore more dynamic adaptation mechanisms based on individual resident needs and preferences.

Summary updated

Abstract

There are considerable advancements in medical health care in recent years, resulting in a rising older population. As the workforce for such a population is not keeping pace, there is an urgent need to address this problem. Having robots to stimulating recreational activities for older adults can reduce the workload for caretakers and give them time to address the emotional needs of the elderly. In this paper, we investigate the effects of the humanoid social robot Nadine as an activity host for the elderly. We propose to evaluate this by placing Nadine humanoid social robot in a nursing home as a caretaker where she hosts bingo game. We record sessions with and without Nadine to understand the difference and acceptance of these two scenarios. We use computer vision methods to analyse the activities of the elderly to detect emotions and their involvement in the game. Our results present positive enforcement during recreational activity, Bingo, in the presence of Nadine. This research is in line with all ethical recommendations as shown in our Annex.
BibTeX
@inproceedings{mishra2021does,
  author = {Nidhi Mishra and Gauri Tulsulkar and Hanhui Li and Nadia Magnenat-Thalmann and Lim Hwee Er and Lee Mei Ping and Cheng Siok Khoong},
  title = {{Does Elderly Enjoy Playing Bingo with a Robot? A Case Study with the Humanoid Robot Nadine}},
  booktitle = {CGI},
  year = {2021},
  url = {https://openreview.net/forum?id=OuynKutJlW9},
}
Figure