TL;DR
This study explores how teachers perceive Augmented Reality (AR)-enhanced analytics (AREA) as a tool for identifying, mitigating, and measuring student disengagement. The findings suggest that while teachers see potential in AREA, its effectiveness is influenced by pedagogical approaches and teaching levels, with a notable distinction in how English teachers build positive relationships.
What Problem Does This Paper Address?
Managing student disengagement in classrooms is a persistent challenge for educators. Each student is unique, and identifying and addressing disengagement can be stressful and time-consuming. Traditional methods like classroom observations and surveys, while useful, often lack real-time data and can be influenced by subjective biases. The educational landscape is constantly evolving, and new technologies like Augmented Reality (AR) offer promising avenues for capturing engagement metrics. However, there's a need to understand how teachers perceive and utilize AR-enhanced analytics (AREA) as a tool for measuring and mitigating student disengagement, especially compared to conventional methods.
Methods
The study employed a mixed-methods, multisession design to gather both qualitative and quantitative data. The process involved four main stages:
- Stage 1: Focus Groups: Small groups of teachers (2-3) shared their strategies for identifying, mitigating, and reporting student disengagement.
- Stage 2: AR App Use: Teachers were introduced to three AR applications designed for educational settings. Their feedback and perspectives on how AR analytics could be used were collected. These AR applications were developed with AREA capabilities, enabling the capture and measurement of user experiences through device sensors (e.g., game scores, screen interactions, movement data, duration, orientation).
- Stage 3: Identify Themes: Themes from the focus groups and AR app sessions were coded to develop a feedback questionnaire.
- Stage 4: Feedback Questionnaire: A larger group of teachers (n=50) completed a questionnaire to rank the perceived importance of various strategies and methods.
The AR applications used were:
- Sheep Shearing AR App: An interactive game for learning about wool and the shearing process.
- Forest Adventure AR Book App: An AR storybook designed to support literacy development and teach abstract principles of sustainability.
- Garden of Eden AR App: An immersive virtual digital space for exploration.
Datasets
The study did not introduce new benchmark datasets. Instead, it utilized data collected from:
- Focus Group Sessions: Qualitative data from 8 primary school teachers.
- AR Application User Experience Data: Sensor data from mobile devices used with the AR applications.
- Post-session Feedback Questionnaire: Quantitative data from 50 teachers.
Key Insights and Ideas
- AR as a Potential Tool: Teachers generally agree that AR applications can generate analytics that help identify disengaged students and that AR can increase learning outcomes and keep students engaged.
- Data is Key: For AR to effectively identify, mitigate, and measure engagement, useful data needs to be collected, analyzed, and presented. This requires integrating mobile device sensors.
- Teacher-Level Differences: There are notable differences in perspectives based on teaching levels, particularly concerning conversations and the use of technology devices. Prep teachers showed higher agreement on conversations, while high school teachers agreed more on technology device use.
- Subject-Specific Pedagogy Matters: While no conclusive differences were found based on teaching subjects, English teachers showed a distinction in building positive relationships, suggesting that subject-specific pedagogy might influence the perceived effectiveness of AREA.
- "Time on Tasks" and "Feedback" are Crucial Metrics: Teachers strongly believe that "Time on Tasks" and "Feedback" are the most effective metrics for measuring student disengagement.
Key Results
- Mitigating Disengagement: Movement and technology were identified as the most popular and effective techniques for mitigating disengagement, with 80.0% of teachers strongly agreeing on their efficacy. Conversations and rewards also received high ratings.
- Measuring Disengagement: "Time on Tasks" (75.0% of teachers strongly agreed) and "Feedback" (60.0% of teachers strongly felt it was a reliable indicator) were deemed the most crucial metrics for measuring student engagement.
- AREA's Potential: Teachers expressed a strong agreement (high mean scores) on the positive impact of AR in areas like conversations, movement, positive relationships, technology use, collaboration, and discovery/exploratory learning.
- Statistical Differences: A statistically significant difference was found between grades and positive relationships, with English teachers showing the highest mean score. Differences were also noted in conversations and the use of technology devices across teaching levels.
Limitations and Future Work
- Sample Size and Generalizability: The qualitative findings were primarily drawn from a small group of eight primary school teachers, which may limit generalizability to broader educator populations. The sample size for the quantitative analysis was 50 teachers.
- Technological Familiarity: Varying degrees of technological familiarity among educators might have influenced their perceptions.
- Nuances in Quantitative Data: The mixed-methods approach, while holistic, might not have captured all specific nuances within the quantitative data.
- Future Research: The authors suggest future research directions including engaging a diverse educator sample across varied educational settings and conducting longitudinal studies to gauge the long-term impact and use of AR analytics and preferred AR applications. They also emphasize that AR must be combined with traditional teaching methods and educators must be adequately trained.
Summary updated