Investigating dialogic interaction in K12 online one-on-one mathematics tutoring using AI and sequence mining techniques

Deliang Wang, Dapeng Shan, Ran Ju, Ben Kao, Chenwei Zhang, Gaowei ChenEduc. Inf. Technol 2025 · DOI

Introduction

The rapid expansion of online one-on-one tutoring, accelerated by the global pandemic, has created a wealth of digital educational data. However, the specific "dialogic" dynamics—the conversational exchanges between tutors and students—have largely remained a "black box." Unlike traditional classrooms where one teacher manages many students, the one-on-one setting offers a unique interpersonal space. Researchers at the University of Hong Kong have addressed this gap by developing an AI-driven pipeline to analyze these interactions at scale, focusing on mathematics tutoring for K-12 students.

Methodology

To analyze over 150,000 messages, the research team built a multi-stage computational pipeline designed to bridge the gap between manual labor and large-scale data analysis:

  • Coding Scheme Development: Utilizing the ICAP framework (Interactive, Constructive, Active, Passive), they created a two-tier scheme consisting of 6 categories and 15 dialog acts. This allows for a granular distinction between simple acknowledgments and deep mathematical reasoning.
  • AI-Powered Annotation: The team fine-tuned a RoBERTa (Robustly Optimized BERT approach) model to automate message labeling.
  • Data Augmentation with ChatGPT: To address "class imbalance"—where critical but rare dialog acts like "reasoning questions" are scarce—the researchers used ChatGPT to generate synthetic contextual utterances, significantly improving the model's accuracy.
  • Lag Sequential Analysis (LSA): This technique was used to identify statistically significant behavioral patterns, mapping how specific tutor actions (e.g., asking a question) influence subsequent student responses.

Dataset Overview

The study analyzed a deidentified mathematics tutorial dataset from a major Singaporean online platform, covering sessions from September 2018 to August 2020. The dataset included 13,796 total sessions.

Table 1: Overview of the online one-on-one math tutoring dataset across different school stages.

As shown in Table 1, the dataset spans Primary, Middle, and High School levels, with the majority of sessions occurring at the Middle School stage. A "gold standard" subset of 1,000 sessions (9,257 messages) was manually annotated by experts to provide a baseline for training and validation.

Key Insights and Discussion

The analysis revealed several critical trends in the digital tutoring landscape:

The "Didactic Trap"

Across all grade levels, tutors dominated the discourse, contributing between 63% and 77% of all messages. The prevailing "provide-and-explain" model suggests that many tutors rely on traditional didactic instruction, which may inadvertently limit opportunities for students to engage in higher-order thinking.

The High School Shift

High school sessions emerged as a distinct outlier. These students were significantly more engaged, sending twice as many messages as their younger counterparts. Crucially, high schoolers were the only group to frequently respond to tutors with independent reasoning rather than simple agreement, indicating a shift toward student-driven learning.

The Role of Social Interaction

Interestingly, "off-topic" or social chatting accounted for over 30% of all interactions. Rather than being a distraction, the researchers argue that this "social glue" is vital for building rapport and creating a secure learning environment, particularly for younger primary students who may need more emotional scaffolding.

Scaffolding Challenges

The study found that tutors rarely posed cognitively challenging questions. When faced with difficult concepts, students often responded by admitting a lack of understanding. This suggests a need for better tutor training in digital scaffolding techniques to help students navigate complex mathematical problems through text-based chat.

Results and Performance

The RoBERTa-based annotator achieved an impressive F1 score of 0.89, outperforming human annotators in consistency. The inclusion of ChatGPT-augmented data was pivotal, as models trained without it achieved a lower F1 score of 0.86.

In terms of engagement:

  • Primary and Middle School: 25-30% of sessions were characterized by "passive" engagement, where student contributions were minimal.
  • High School: Passive engagement dropped to only 8.2%, reflecting a much more balanced and interactive tutor-to-student ratio.

Conclusion and Future Directions

While the study provides a robust framework for analyzing online tutoring, it is limited by its geographic focus on Singapore and its reliance on "lag-1" transitions in LSA. Future research could explore longer "chains" of thought and more diverse cultural contexts.

The ultimate goal of this work is the development of real-time "tutor dashboards." By providing live feedback on dialogic patterns, such tools could alert tutors when they are dominating the conversation or failing to elicit student reasoning, leading to more effective and interactive online education.

Summary updated

BibTeX
@article{wang2025investigating,
  author = {Deliang Wang and Dapeng Shan and Ran Ju and Ben Kao and Chenwei Zhang and Gaowei Chen},
  title = {{Investigating dialogic interaction in K12 online one-on-one mathematics tutoring using AI and sequence mining techniques}},
  journal = {Educ. Inf. Technol},
  year = {2025},
  doi = {10.1007/s10639-024-13195-9},
}
Figure