Tracking Knowledge Structures and Proficiencies of Students With Learning Transfer

Hengyu Liu, Tiancheng Zhang, Fan Li, Yu Gu, Ge Yu

Abstract

In online intelligent education, Knowledge Tracing (KT) is essential for personalized learning. However, traditional models often focus solely on knowledge proficiency, ignoring the critical role of learning transfer—the process where mastering one concept facilitates the acquisition of another. This paper proposes the Knowledge Tracing with Learning Transfer (KTLT) model. KTLT moves beyond shallow metrics by tracking a student's abstract principles and evolving knowledge structures. By incorporating educational psychology theories, the model achieves superior predictive accuracy and provides interpretable insights into a student's mental map.

Personalized learning platforms like MOOCs rely on Knowledge Tracing (KT) to estimate a student's mastery of specific concepts. While existing models are effective at calculating numerical proficiency scores, they often fail to capture the "how" and "why" of learning. Specifically, they overlook learning transfer: the phenomenon where underlying principles or related concepts accelerate the learning of new material.

Conceptual illustration of learning transfer: Student B's prior experience with C++ makes learning Python easier compared to Student A, who lacks a related programming background.

As shown in Figure 1, a student with a background in C++ (Student B) will likely find Python easier to master than a student with no programming experience (Student A). This is because Student B has already developed an "abstract principle mastery" of programming logic that transfers across languages. Current KT models struggle to account for this mental map, or "knowledge structure," that students build over time.

The KTLT Framework

To address these limitations, the authors propose KTLT, a multilevel model that integrates educational priors with a probabilistic matrix factorization framework. Unlike traditional "black-box" deep learning models, KTLT is designed around three distinct layers of student tracking:

  1. Abstract Principle Mastery: Tracks common attributes across concepts, such as logical reasoning or general learning habits.
  2. Knowledge Proficiency: Measures the mastery of explicit, expert-defined concepts.
  3. Knowledge Structure: Captures the perceived similarities and differences between concepts as the student understands them.
The multilevel architecture of the KTLT model, showing the flow from abstract principles to problem mastery and final student response.

The model utilizes several key matrices to guide its learning:

  • Q-matrix: Maps problems to specific knowledge concepts (a standard in the field).
  • D-matrix: Maps knowledge concepts to broader abstract principles.
  • S-matrix: Represents the correlation level between different knowledge concepts.

Furthermore, KTLT incorporates dynamic temporal factors, including the learning curve (knowledge gain) and the Ebbinghaus forgetting curve (knowledge decay), to ensure the model reflects the reality of human memory.

Experimental Evaluation

The KTLT model was evaluated using two real-world datasets from online programming judges: HDU (HDU OnlineJudge) and POJ (PKU OnlineJudge). These datasets are ideal for tracking learning transfer because they record repeated submissions, allowing the model to observe how a student's understanding evolves through trial and error.

Key Results

  • Superior Accuracy: KTLT consistently outperformed seven state-of-the-art baselines, including both traditional models (BKT, IRT) and deep learning approaches (EERNN). It achieved the lowest Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) across both datasets.
  • Interpretability: A major advantage of KTLT is its transparency. Unlike deep learning models, it can generate diagnostic visualizations, such as radar charts of mastery and network graphs of a student's knowledge structure. This allows educators to see the "darker, wider lines" of connection that experts possess but beginners lack.
  • Ablation Success: Experiments showed that removing components like "Deep Knowledge Structure" or the "Q-matrix priors" significantly degraded performance, proving that both abstract principles and expert-defined concept mappings are essential.

Discussion and Future Work

KTLT represents a significant step toward more cognitively grounded AI in education. By applying Cognitive Structure Migration Theory, the research demonstrates that learning transfer depends on the availability, stability, and discernibility of a student's existing knowledge structure.

However, some limitations remain. The current model is designed for batch processing rather than real-time tracking. Future iterations may adopt Hidden Markov Model-based approaches to enable live feedback. Additionally, while the model identifies "abstract principles," these remain latent factors; future work aims to map these to specific educational psychology constructs like "mathematical logic" or "algorithmic thinking" for even greater clarity.

Summary updated

Abstract

In online intelligent education systems, to offer proactive studying services to students (e.g., learning path recommendation), a crucial demand is to track students' knowledge mastery levels over time. However, existing works ignore the impact of learning transfer on knowledge tracing and only track knowledge proficiency. Knowledge proficiency alone cannot fully reflect students' knowledge mastery levels. A student's knowledge structure (the similarities and differences within knowledge concepts) and abstract principle mastery level (common attributes among knowledge concepts, such as learning methods) also need to be tracked. To this end, we propose a novel multilevel Knowledge Tracing model with Learning Transfer (KTLT) to track students' knowledge mastery levels. First, we clarify the relationships among abstract principles, knowledge structure, and knowledge proficiency by utilizing the learning transfer theory in educational psychology. Then, we associate each problem with a knowledge vector in which each element represents an explicit knowledge concept by leveraging educational priors (i.e., a Q-matrix). Correspondingly, each student is represented as a knowledge vector (knowledge proficiency) and an abstract principle vector (abstract principle mastery). Given a student's knowledge and abstract principle vector over time, we use the learning and forgetting curve as priors to capture the student's knowledge proficiency and abstract principle mastery level over time. Furthermore, we embed the knowledge concept by using a student's abstract principle mastery level and obtain a personalized knowledge relevance matrix (a student's knowledge structure) by calculating the cosine similarity among the knowledge embedding results. Finally, we design a probabilistic matrix factorization framework by combining student and problem priors for tracking a student's knowledge mastery. Extensive experiments on two real-world datasets demonstrate both the effectiveness and explanatory ability of the KTLT.
BibTeX
@article{liu2021tracking,
  author = {Hengyu Liu and Tiancheng Zhang and Fan Li and Yu Gu and Ge Yu},
  title = {{Tracking Knowledge Structures and Proficiencies of Students With Learning Transfer}},
  journal = {IEEE Access},
  year = {2021},
  doi = {10.1109/access.2020.3032141},
}
Figure