Abstract
Concept prerequisite learning (CPL) plays a key role in developing technologies that assist people to learn a new complex topic or concept. Previous work commonly assumes that all concepts are given at training time and solely focuses on predicting the unseen prerequisite relationships between them. However, many real-world scenarios deal with concepts that are left undiscovered at training time, which is relatively unexplored. This paper studies this problem and proposes a novel alternating knowledge distillation approach to take advantage of both content- and graph-based models for this task. Extensive experiments on three public benchmarks demonstrate up to 10% improvements in terms of F1 score.
Introduction
In educational technology, Concept Prerequisite Learning (CPL) is the task of automatically determining if one concept (e.g., "Linear Algebra") is a necessary precursor to another (e.g., "Principal Component Analysis"). Most existing CPL models operate in an "offline" or "closed-world" setting, assuming all possible concepts are known during training. However, in dynamic real-world environments, new topics emerge constantly. Traditional graph-based models struggle with these "unseen" concepts because they rely on fixed node embeddings. This paper introduces the task of CPL for unseen concepts, where the system must predict relationships for concepts it has never encountered before.
Methodology
The authors propose a dual-component architecture coordinated by a novel training strategy called Alternating Knowledge Distillation (AKD).
The framework consists of three main parts:
- Retrieval-Augmented Content Model ($\theta_C$): Since concept names are often too brief, the model uses BM25 to retrieve the most relevant 100-token passage from Wikipedia for every concept. These descriptions, along with concept names, are fed into a BERT model to classify prerequisite relationships.
- Graph-Based Model ($\theta_G$): This component uses Neural Collaborative Filtering (NCF) to treat prerequisite prediction as a link prediction problem, adapted for directed graphs.
- Alternating Knowledge Distillation (AKD): To solve the "cold start" problem for unseen nodes in the graph model, the content model (which can process any text) acts as a "teacher." The content model is first trained on ground truth and generates "pseudo-labels" for the graph model. The roles then flip, with the graph model refining the content model. This iterative cycle allows for the transfer of knowledge between textual semantics and structural patterns.
The final prediction is a weighted linear interpolation of the scores from both models.
Experimental Setup
The approach was evaluated on three manually annotated public benchmarks:
- University Course: CS course descriptions from 11 US universities.
- LectureBank: Online lecture files covering NLP and related topics.
- MOOC ML: Prerequisite relations from Coursera machine learning video subtitles.
To simulate the "unseen" scenario, 10% of concepts (and all their associated links) were randomly held out from the training set to serve as the test set.
Results and Discussion
The proposed model significantly outperformed competitive baselines across all datasets, demonstrating the effectiveness of the AKD framework.
Key Findings:
- State-of-the-Art Performance: As shown in the results table, the model achieved an F1 score of 0.842 on the University Course dataset, a substantial improvement over the BERT baseline (0.767).
- Importance of External Knowledge: Augmenting the model with Wikipedia descriptions proved crucial. Concept names alone are often insufficient for deep semantic understanding.
- Structural Logic: While the content model is stronger for unseen concepts, the graph model helps "correct" it by enforcing structural logic (e.g., transitivity in prerequisites).
- Ablation Success: Removing the AKD process led to a significant drop in performance, confirming that iterative teacher-student training is more effective than simple model averaging.
Conclusion
This research addresses the critical "cold start" problem in Concept Prerequisite Learning. By leveraging retrieval-augmented text models and graph-based collaborative filtering through an alternating distillation framework, the system can effectively generalize to concepts not seen during training. Future work includes automating concept extraction from raw text and scaling the system to handle the tens of thousands of concepts found on large-scale MOOC platforms.
Summary updated