Virtual Reality for Screening of Cognitive Function in Older Persons: Comparative Study

Sean Ing Loon Chua, Ngiap Chuan Tan, Wei Teen Wong, John Carson Allen, Joanne Hui Min Quah, Rahul Malhotra, Truls ØstbyeJournal of Medical Internet Research 2019 · DOI

TL;DR

This study explores the feasibility and acceptability of using a virtual reality (VR) system, the RE@CH assessment module, for screening cognitive function in older adults in a primary care setting. The VR tool proved to be feasible, acceptable to participants, and demonstrated a moderate ability to discriminate between cognitively intact and impaired individuals, suggesting its potential as a novel screening method.

What Problem Does This Paper Address?

Dementia is a growing global health concern, characterized by cognitive decline in various domains. Traditional screening tools, like the MMSE and MoCA, primarily focus on memory and executive function, often overlooking other crucial cognitive areas such as perceptual-motor skills. Furthermore, these paper-and-pencil or digitalized tests can be influenced by educational and cultural backgrounds. The authors aim to address these limitations by investigating the use of virtual reality (VR) technology, which offers a more immersive and ecologically valid approach to assessing a broader range of cognitive functions, particularly in a primary care setting where early detection is crucial.

Methods

The study employed a comparative design at a public primary care clinic. Participants aged 65-85 were recruited and divided into two groups based on their Montreal Cognitive Assessment (MoCA) scores: Group 1 (cognitively intact, MoCA ≥ 26) and Group 2 (cognitively impaired, MoCA < 26).

The core of the study involved the RE@CH assessment module, a VR system utilizing motion sensor (Leap Motion) technology. This module presents participants with simulated daily living activities as 3D games displayed on a 55-inch screen. The VR environment was designed to assess learning and memory, perceptual-motor function, and executive function. A scoring algorithm was developed to evaluate participant performance based on task completion time, number of attempts, and accuracy.

To assess feasibility and acceptability, researchers measured the percentage of participants who successfully completed the module, the time taken for completion, and collected feedback through a questionnaire.

For performance and validity, the study compared the total performance scores from the RE@CH module between the two groups. They also examined the correlation of these scores with established cognitive screening tools (Abbreviated Mental Test - AMT, Mini-Mental State Examination - MMSE, and MoCA). Receiver Operating Characteristic (ROC) curve analysis was used to determine the module's predictive capability.

Datasets

Not discussed in the paper. The study utilized a custom-developed VR assessment module (RE@CH) and standard cognitive screening tests (AMT, MMSE, MoCA) for evaluation.

Key Insights and Ideas

  • VR as a Promising Screening Tool: The study strongly suggests that VR can be a feasible and acceptable method for cognitive screening in older adults within a primary care setting, offering a more comprehensive assessment than traditional tools.
  • Ecological Validity of VR: The use of simulated daily living tasks in VR provides a more realistic assessment of cognitive function compared to abstract paper-and-pencil tests.
  • Acceptance by Older Adults: Participants generally responded positively to the VR experience, indicating a willingness to embrace new technologies for healthcare.
  • Discrimination Power Varies by Task: While the overall VR performance score differentiated between groups, specific tasks were more effective than others, highlighting the need for careful task selection and algorithm refinement.
  • Moderate Correlation with Existing Tools: The VR module's performance scores showed moderate positive correlations with established cognitive tests, suggesting it captures similar cognitive aspects but also unique ones.

Key Results

  • Feasibility: All 60 recruited participants (37 in Group 1, 23 in Group 2) successfully completed the VR assessment module.
  • Time to Completion: The mean time to complete the VR module was comparable to traditional screening tools, averaging 19.1 minutes for the cognitively intact group and 20.4 minutes for the impaired group.
  • Acceptability: Participants provided favorable feedback scores (ranging from 3.80 to 4.48 out of 5) regarding their experience with the VR module.
  • Performance Discrimination: The cognitively intact group (Group 1) achieved significantly higher total performance scores (552.0) compared to the cognitively impaired group (Group 2) (476.1, P<.001). Tasks 2 (making a phone call), 3 (reading newspaper), and 7 (shopping) showed the most significant differences between groups.
  • Validity: The total performance score from the VR module showed moderate positive correlations with AMT (0.312), MMSE (0.373), and MoCA (0.427).
  • ROC Analysis: The ROC curve analysis indicated a strong ability of the VR module's total performance score to discriminate between cognitively intact and impaired individuals, with an Area Under the Curve (AUC) of 0.821. An optimal cutoff score of 500 yielded 78.2% sensitivity and 75.7% specificity.

Limitations and Future Work

The authors acknowledge several limitations:

  • Educational Disparity: A significant difference in educational levels between the two groups could have influenced performance.
  • Small Sample Size: The study was a feasibility study with a relatively small sample size, limiting generalizability.
  • Recruitment Location: Recruitment was limited to a single location, potentially affecting the representativeness of the sample.
  • Screening Tool vs. Diagnostic Tool: The MoCA is a screening tool, not a diagnostic one, meaning some participants with subjective cognitive impairment but normal MoCA scores might have been misclassified.
  • Prototype Nature of VR Module: The RE@CH module was a prototype adapted from rehabilitation, and some tasks might have been challenging due to design rather than cognitive impairment.

Future work suggested includes:

  • Larger, Randomized Controlled Trials: Conducting adequately powered trials, stratified by education level, to further assess VR's utility.
  • Development of a Dedicated VR Screening Program: Creating a new VR program specifically designed for cognitive function assessment to address the limitations of the current prototype.
  • Eliminating Non-Differentiating Tasks: Refining the VR module by removing tasks that do not effectively differentiate between cognitive groups and optimizing time efficiency.
  • Incorporating Time Motion Measurements: Evaluating accuracy, efficiency, and cost-effectiveness in comparison to conventional tools.

Summary updated

Abstract

We demonstrated the feasibility of using a VR-based screening tool for cognitive function in older persons in primary care, who were largely in favor of this tool.
BibTeX
@article{chua2019virtual,
  author = {Sean Ing Loon Chua and Ngiap Chuan Tan and Wei Teen Wong and John Carson Allen and Joanne Hui Min Quah and Rahul Malhotra and Truls Østbye},
  title = {{Virtual Reality for Screening of Cognitive Function in Older Persons: Comparative Study}},
  journal = {Journal of Medical Internet Research},
  year = {2019},
  doi = {10.2196/14821},
}
Figure