Introduction
Prerequisite learning is the task of automatically identifying whether one concept must be mastered before another (e.g., understanding "Addition" before "Multiplication"). This paper details the B4DS team's participation in the EVALITA 2020 PRELEARN shared task, which focused on classifying prerequisite relationships between Italian Wikipedia concepts across four educational domains: Physics, Precalculus, Data Mining, and Geometry.
Determining these relationships is a complex task that typically requires deep domain expertise. Computational solutions offer significant potential for educational technology, including the automated quality assessment of curricula and the generation of optimized learning paths through large knowledge bases like Wikipedia.
Methodology
The authors focused on the raw-text setting of the challenge, comparing two distinct architectural approaches to knowledge extraction:
- Model 1 (XGBoost + Handcrafted Features): This ensemble classifier utilizes a 614-dimensional feature vector. It combines 300-dimensional Italian GloVe word embeddings (averaged from concept titles) with 14 handcrafted textual features, such as Jaccard similarity, word occurrence counts, and length differences between introductory paragraphs.
- Model 2 (Gated Recurrent Unit - GRU): A neural network approach that processes the first 400 words of each Wikipedia page. It employs a small GRU (hidden size 8, encoding size 32) to generate representations for page pairs, which are then concatenated and passed to a linear classification layer.
Experimental iterations involving Graph Convolutional Networks (GCNs) to leverage Wikipedia's hyperlink structure were discarded after they systematically degraded performance. Large transformer models like BERT were also avoided due to the small dataset size, which led to significant overfitting.
Experimental Results
The models were evaluated in two settings: In-domain (training and testing on the same domain) and Cross-domain (training on three domains and testing on the fourth). The B4DS team secured second place in the PRELEARN challenge.
As shown in Figure 1, Model 1 consistently outperformed Model 2, particularly in the cross-domain setting where generalization is critical. While Model 2 remained competitive in in-domain tasks, its performance dropped significantly when applied to unseen domains, likely due to the GRU's reliance on specific textual patterns that did not generalize as well as the broader knowledge captured by GloVe embeddings.
Detailed analysis in Figure 2 reveals that Model 1 generally achieved higher recall, leading to superior F1 scores across most domains. In contrast, Model 2 occasionally demonstrated higher precision but missed a larger number of positive prerequisite samples. In domains like Physics and Precalculus, both models showed a wider gap between precision and recall, suggesting that prerequisite concepts in these fields are less explicitly described in general Wikipedia text.
Discussion and Conclusion
The primary insight from this study is that simpler ensemble models can outperform deep learning architectures when working with small, domain-specific datasets. The success of Model 1 is attributed to the use of pre-trained GloVe embeddings, which inject external world knowledge that the task-specific dataset lacks. Conversely, the GRU struggled to capture the complex, long-distance relationships inherent in prerequisite identification from limited text.
Future work will focus on addressing data scarcity by combining task-specific data with the broader Italian Wikipedia corpus. Additionally, the authors propose exploring hybrid architectures that combine statistical bag-of-words features with recurrent models to capture both bottom-up statistics and sequential patterns simultaneously.
Summary updated