Immersive virtual environments and embodied agents for e-learning applications

Isabel S. Fitton, Daniel J. Finnegan, Michael J. ProulxPeerJ Computer Science 2020 · OpenReview

TL;DR

This study investigates the effectiveness of immersive virtual environments (IVEs) for informational e-learning, comparing them to traditional classrooms. The findings suggest that IVEs can deliver comparable learning outcomes and engagement levels, even without embodied agent tutors, offering a scalable alternative to current distance learning platforms.

What Problem Does This Paper Address?

Massive Open Online Courses (MOOCs) have democratized access to education, but they struggle with low learner commitment and completion rates. Traditional e-learning often lacks the engagement and immersion of in-person learning. This paper explores whether Immersive Virtual Environments (IVEs), specifically those mimicking a classroom lecture, can bridge this gap by providing a more engaging and effective learning experience for informational content, and whether embodied agents (virtual tutors) enhance this experience.

Methods

The researchers employed a between-participant design, randomly assigning participants to one of four learning conditions:

  1. Non-virtual learning environment: A traditional seminar room with a PowerPoint presentation and a human experimenter acting as a tutor.
  2. IVE with no tutor: A virtual replica of the classroom without any embodied agent.
  3. IVE with a humanoid tutor: The virtual classroom with a 3D avatar of a female tutor.
  4. IVE with an abstract tutor: The virtual classroom with a simple, geometric-shaped animated agent.

Learning material consisted of novel information about a fictional alien species to control for prior knowledge. Performance was measured by a knowledge test (negatively marked to discourage guessing), and learner motivation, satisfaction, and engagement were assessed via questionnaires. Virtual presence was also measured. Statistical analysis included Kruskal-Wallis tests for comparing multiple groups and independent t-tests for comparing two groups, supplemented by Bayesian statistics (Bayes Factors) to assess the strength of evidence for or against the null hypothesis.

Datasets

The study did not use pre-existing datasets for training or evaluation. Instead, a bespoke knowledge test was created to assess learning of novel material about a fictional alien species. This material was specifically designed to eliminate the confounding variable of prior knowledge.

Key Insights and Ideas

  • IVEs are as effective as traditional classrooms for informational learning: The study found no significant difference in learner performance between participants learning in a virtual classroom and those in a real-world classroom. This is a crucial finding for scalability, as IVEs can potentially reach far more students than physical classrooms.
  • Embodied agents don't necessarily boost performance in this context: Contrary to some expectations, the presence of either a humanoid or abstract embodied agent tutor did not lead to significantly better performance compared to learning without a tutor in the IVE. This challenges the assumption that virtual tutors are always beneficial for informational learning.
  • Learner satisfaction and engagement are comparable across IVE conditions: Participants reported similar levels of satisfaction and engagement whether a tutor was present or not, and regardless of the tutor's appearance (humanoid vs. abstract).
  • The "uncanny valley" might be a factor for humanoid avatars: Participants in the humanoid tutor condition reported finding the avatar "strange," citing issues with repetitive movement and lack of facial expression. This suggests that a mismatch between expected and actual avatar behavior can negatively impact perception, potentially hindering engagement.
  • Abstract tutors might be less prone to negative perceptions: The abstract tutor, lacking human-like features, did not elicit the same negative reactions, suggesting it might be a safer choice when realism is not perfectly achieved.

Key Results

  • Learner Performance: No statistically significant differences were found in test scores across the four learning conditions (Non-virtual, IVE No Tutor, IVE Humanoid Tutor, IVE Abstract Tutor). The Bayes Factor indicated that the data were six times more likely to be explained by the null hypothesis (no difference).
  • Learner Satisfaction & Engagement: Similar satisfaction and engagement scores were observed across all virtual learning conditions, with slightly higher levels in the IVE with no tutor. The Bayes Factor suggested the data were four times more likely to support the null hypothesis.
  • Learner Motivation: While motivation scores were slightly higher for the abstract tutor condition, the difference was not statistically significant. The Bayes Factor indicated the data were almost equally likely to be explained by the null or alternative hypothesis.
  • Virtual Presence: Ratings of virtual presence were consistent across all IVE conditions and did not differ significantly. Bayes statistics provided moderate support for the null hypothesis.
  • Learner Perceptions of Avatar: 58% of participants in the humanoid tutor condition found the avatar "strange" due to lack of facial expression and repetitive movement.

Limitations and Future Work

The authors acknowledge several limitations:

  • Short-term assessment: The study only measured immediate post-learning outcomes. Future work should include longitudinal studies to assess the long-term retention and impact of IVE learning.
  • Participant recruitment: The study was limited to university students, which may not represent the broader population. Future studies should consider a wider range of participants and use random allocation to minimize individual differences.
  • Avatar realism: The humanoid avatar lacked facial animations, which likely contributed to negative perceptions. Future research should explore the impact of more sophisticated avatar designs, including changing expressions and verbal interaction.
  • Relevance of material: The study used fabricated material. Future work could investigate how the relevance of the learning material to the tutor's appearance and form influences learning outcomes.

The authors suggest that future research should explore the impact of EA's changing expressions, verbal interaction, and the contextual relevance of the learning material to the EA's appearance. They also propose investigating the potential trend that younger participants might be more influenced by EAs.

Summary updated

Abstract

Massive Open Online Courses are a dominant force in remote-learning yet suffer from persisting problems stemming from lack of commitment and low completion rates. In this initial study we investigate how the use of immersive virtual environments for Power-Point based informational learning may benefit learners and mimic traditional lectures successfully. We examine the role of embodied agent tutors which are frequently implemented within virtual learning environments. We find similar performance on a bespoke knowledge test and metrics for motivation, satisfaction, and engagement by learners in both real and virtual environments, regardless of embodied agent tutor presence. Our results raise questions regarding the viability of using virtual environments for remote-learning paradigms, and we emphasise the need for further investigation to inform the design of effective remote-learning applications.
BibTeX
@inproceedings{fitton2020immersive,
  author = {Isabel S. Fitton and Daniel J. Finnegan and Michael J. Proulx},
  title = {{Immersive virtual environments and embodied agents for e-learning applications}},
  booktitle = {PeerJ Computer Science},
  year = {2020},
  url = {https://openreview.net/forum?id=Liat4YwrqV},
}
Figure