Introduction
In educational technology, Knowledge Tracing (KT) serves as a cornerstone for estimating student mastery. Traditional KT models typically rely on a linear sequence of question-response pairs to predict future performance. However, this approach often overlooks the nuanced learning process that occurs during reading. In online platforms, students engage in Self-Regulated Learning (SRL) behaviors—such as highlighting text, making annotations, and looking up vocabulary—which provide rich signals of comprehension and engagement.
The challenge lies in the heterogeneous nature of these actions; a "highlight" is semantically different from a "multiple-choice answer." This paper introduces a transformer-based model designed to unify these diverse interactions into a single predictive framework.
Methodology
The researchers proposed a transformer architecture that treats all student interactions as temporal events. To bridge the gap between different action types, they utilized the Universal Sentence Encoder (USE) to create semantic embeddings for question text, student notes, and highlighted snippets.
A key innovation is the Relation-Aware Attention mechanism, which incorporates three specific coefficients:
- Time Relation: Accounts for human forgetting by weighting recent actions more heavily.
- Question Relation: Measures semantic similarity between the current and previous questions.
- Response-SRL Relation: Quantifies the semantic link between a student's prior SRL behaviors (like notes) and the current assessment question.
Experimental Setup
The model was evaluated using data from Actively Learn, a K-12 online reading platform. The study focused on middle school physical science students across three curated datasets:
- D1 (No SRL): A baseline containing only question attempts.
- D2 (SRL): A comprehensive dataset including both questions and SRL behaviors.
- D2.1 (Removed SRL): A controlled subset of D2 where SRL actions were stripped to isolate their impact.
The model was tested on both classification (predicting a perfect score) and regression (predicting a specific score from 0-4) tasks.
Key Results and Insights
The inclusion of SRL behaviors significantly enhanced predictive accuracy. On the D2 dataset, the model achieved an AUC of 0.85 and an accuracy of 0.78, outperforming state-of-the-art baselines like Deep Knowledge Tracing (DKT).
Ablation studies revealed that the "Response-SRL Relation" is a critical feature; removing the semantic link between study habits and questions led to a notable drop in performance. This confirms that reading behaviors are not merely "noise" but are highly predictive of learning outcomes. Furthermore, the model's attention mechanism provides interpretability, allowing educators to see which specific notes or highlights most influenced a student's predicted success.
Conclusion and Future Work
By treating reading as an active learning process rather than a passive precursor to assessment, this research demonstrates that SRL behaviors are vital for accurate student modelling. Future work aims to refine the time-decay functions and explore how these predictive insights can be used to trigger real-time pedagogical interventions, such as suggesting specific reading strategies to struggling students.
Summary updated