Learning states enhanced Knowledge Tracing: Simulating the diversity in real-world learning process

Shanshan Wang, Xueying Zhang, Xun Yang, Xingyi Zhang, Keyang WangExpert Systems with Applications 2025 · DOI · OpenReview

TL;DR

This paper introduces LSKT, a novel Knowledge Tracing (KT) model that enhances prediction accuracy by explicitly modeling both a learner's knowledge state and their dynamic learning state. By incorporating these two factors, LSKT better simulates the complexities of real-world learning, outperforming existing state-of-the-art methods on multiple benchmark datasets.

What Problem Does This Paper Address?

Knowledge Tracing (KT) aims to predict a learner's future performance based on their past interactions with educational content. Traditional KT models, and even recent deep learning approaches like Attention-based DLKT (ATT-DLKT), often struggle to accurately capture the nuances of the learning process. Two main limitations are identified:

  1. Difficulty in identifying relevant historical interactions: Existing models tend to rely heavily on the similarity between exercises. However, factors like exercise difficulty, response reliability (due to guessing), and subtle differences between exercises covering the same concept make it hard to pinpoint the most informative past interactions. This can lead to noise in model training.
  2. Ignoring the learner's dynamic learning state: The learner's overall state of engagement, confidence, or fatigue (referred to as "learning state") significantly influences their performance. This dynamic aspect, which changes over time and is distinct from their knowledge of specific concepts, has been largely overlooked by previous KT models.

LSKT aims to address these limitations by more realistically simulating the learning process, considering both the learner's knowledge and their evolving learning state.

Methods

LSKT builds upon the Transformer architecture and introduces several novel components:

  • IRT-based Feature Embedding Module: Inspired by Item Response Theory (IRT), the paper proposes three levels of granularity for embedding exercise and interaction features:
  • LSKT-1PL: Models coarse-grained differences using a single parameter (difficulty).
  • LSKT-2PL: Extends to two parameters (difficulty and distinctiveness) for sub-fine-grained modeling.
  • LSKT-3PL: Further incorporates a random guessing factor for fine-grained simulation of learner responses. This module aims to capture subtle differences between exercises and interactions, mitigating overfitting and improving interpretability.

  • Learning State Extraction (LSE) Module: This module uses multi-layer causal convolutional layers with varying receptive fields to capture the learner's changing learning state over multiple time scales. It focuses on recent performance patterns rather than just similarity to past exercises.

  • Learning State Enhanced Knowledge State Extraction Module: This is a key innovation where the extracted learning state is integrated into the process of capturing the knowledge state. It uses a sparse attention mechanism, guided by k-means clustering of learning states, to focus on historical moments that are not only similar in terms of exercise but also in terms of the learner's learning state at that time. This helps to obtain a more detailed and accurate knowledge state.
  • Learner Response Prediction Module: The final prediction is made by fusing the enhanced knowledge state and the learning state, along with the embedding of the next exercise.

Datasets

The model was evaluated on four real-world educational datasets:

  • ASSIST09: Collected from the ASSISTment system (2004).
  • ASSIST12: Also from the ASSISTment platform (2012-2013), considered a benchmark.
  • ASSISTChall: From a 2017 data mining competition, featuring longer interaction sequences.
  • algebra05: From the KDDcup 2010 Educational Data Mining Challenge (2005-2006).

The paper notes that ASSIST09 lacks timestamp information, and they followed previous studies to sort sequences by order ID.

Key Insights and Ideas

  • Learning State is Crucial: The paper strongly emphasizes that a learner's dynamic learning state is as important as their knowledge state for accurate prediction. Ignoring it leads to significant performance degradation.
  • Fine-grained Feature Modeling Matters: Using IRT-inspired models to capture differential aspects of exercises (difficulty, distinctiveness, guessing) leads to better performance and interpretability compared to simply using concept embeddings.
  • Sparse Attention for Learning States: Clustering learning states and using sparse attention helps to focus on relevant historical learning moments, preventing noise from irrelevant past states.
  • Complementary Nature of States: Knowledge state and learning state are not redundant; they complement each other, and jointly modeling them yields the best results.

Key Results

  • State-of-the-Art Performance: LSKT consistently outperformed baseline methods (DKT, DKVMN, SAKT, AKT, DTransformer, FKT) across all four datasets in terms of AUC and ACC metrics. For example, it achieved an improvement of up to 3.33% in AUC on ASSISTChall.
  • Ablation Study Findings:
    • Removing either the learning state extraction or the knowledge state extraction module significantly degraded performance.
  • Integrating the learning state into knowledge state extraction (LSKT vs. LSKT-RLE) improved performance.
  • The LSKT-3PL model (with guessing factor) generally performed best, though LSKT-2PL was competitive on datasets with fewer exercise types.
  • Clustering Parameter Optimization: The optimal number of learning state clusters (n) was found to be 4 across all datasets, balancing the inclusion of relevant historical moments and avoiding noise.
  • Visualization Insights: T-SNE visualizations showed that knowledge states were more clearly separated when guided by learning states. Embeddings from IRT-based models showed better clustering and differentiation compared to simple concept embeddings.

Limitations and Future Work

  • Dataset Specificity: The paper notes that the LSKT-3PL model's advantage was less pronounced on the algebra05 dataset, suggesting that for datasets with fewer exercise types, finer-grained modeling might not always be necessary or beneficial.
  • Future Work: The authors plan to explore:
  • Differences between exercises within the same knowledge concept.
  • Connections between exercises across different knowledge concepts.
  • Further enhancing model performance and applicability.

Summary updated

BibTeX
@article{wang2025learning,
  author = {Shanshan Wang and Xueying Zhang and Xun Yang and Xingyi Zhang and Keyang Wang},
  title = {{Learning states enhanced Knowledge Tracing: Simulating the diversity in real-world learning process}},
  journal = {Expert Systems with Applications},
  year = {2025},
  doi = {10.1016/j.eswa.2025.126838},
}
Figure