Study-GNN: A Novel Pipeline for Student Performance Prediction Based on Multi-Topology Graph Neural Networks

Ming Li, Xiangru Wang, Yi Wang, Yuting Chen, Yixuan ChenSustainability 2022 · DOI

TL;DR

This paper introduces Study-GNN, a novel pipeline for predicting student academic performance by leveraging the relationships between students. By constructing multiple "views" of student interactions as graphs and fusing their information using a multi-topology graph neural network with an attention mechanism, Study-GNN significantly improves prediction accuracy over traditional methods and single-graph approaches.

What Problem Does This Paper Address?

Predicting student academic performance is crucial for early intervention, personalized learning, and improving educational outcomes. However, many existing methods treat students in isolation, failing to capture the rich, often implicit, relationships and peer influences that significantly impact academic success. This paper addresses this gap by proposing a framework that explicitly models these student-to-student relationships using graph structures.

Methods

The core of this work is the Study-GNN pipeline, which consists of several key components:

  • Multi-Topology Graph Construction: Instead of relying on a single representation of student relationships, the authors construct multiple graphs. Each graph captures different facets of student interactions based on various similarity learning metrics (Cosine, Pearson, Tanimoto) applied to student attributes. This creates diverse "views" of the student network.
  • Multi-Topology Graph Neural Network (MTGNN): A novel GNN architecture is proposed to process these multiple graph topologies. For each graph, a standard Graph Convolutional Network (GCN) is used to learn node embeddings.
  • Attention-Based Fusion: To effectively combine the information from these different graph topologies, an attention mechanism is employed. This mechanism learns to assign importance weights to each graph's embeddings, allowing the model to dynamically focus on the most relevant relationship views for prediction.
  • Semi-Supervised Node Classification: The entire pipeline is framed as a semi-supervised node classification task, where each student is a node, and their predicted performance (e.g., Pass/Fail/Withdrawal) is the node label.

Datasets

The study utilizes the Open University Learning Analytics dataset (OULA), specifically the code-Mode CCC subset. This dataset contains demographic information, online learning behavior (clickstream data), and learning evaluation data for 3983 students. The authors preprocessed this data to obtain academic records, categorizing students into "Distinction," "Pass," "Fail," and "Withdrawal" groups, which were further combined for specific prediction tasks.

Key Insights and Ideas

  • Student relationships matter: The paper strongly demonstrates that modeling student-to-student relationships is not just a theoretical concept but yields tangible improvements in prediction accuracy.
  • Multiple perspectives are better than one: Using multiple graph topologies (different similarity metrics) provides richer information than a single graph, highlighting the benefit of diverse relationship views.
  • Attention is key for fusion: The attention mechanism effectively learns to weigh the importance of different relationship graphs, allowing the model to adaptively leverage the most informative views for prediction.
  • Early prediction is feasible and impactful: The model shows promise in identifying at-risk students early in the semester, which is crucial for timely interventions.

Key Results

  • Superior Performance: Study-GNN significantly outperformed baseline models, including traditional machine learning methods (SVM, LR, SLFNN) and a single-topology GCN, across various prediction tasks.
  • Accuracy Gains: For predicting final academic performance (Pass/Fail), MTGNN achieved an accuracy of 84.98%, compared to the best baseline (GCN-Pearson) at 81.48%. For Pass/Withdrawal, MTGNN reached 91.95% accuracy, outperforming GCN-Pearson at 87.31%.
  • Early Prediction Effectiveness: In early prediction tasks (e.g., at Week 20), MTGNN achieved 81.14% accuracy for predicting failure and 82.30% for predicting withdrawal, demonstrating its utility for timely interventions.
  • Visual Confirmation: Visualization of node embeddings showed that MTGNN learned more compact and well-separated clusters for different performance categories compared to original embeddings.

Limitations and Future Work

  • Graph Construction Dependency: The performance of MTGNN is tied to the quality and relevance of the constructed student relationship graphs. The paper notes that the similarity learning process can be simplified if node features are noise-free and the graph structure exhibits strong homophily.
  • Computational Complexity: While generally efficient, the complexity can increase with the number of topologies (M). The authors suggest that a small number of topologies is often sufficient, mitigating this concern.
  • Future Directions:
    • Enhanced Graph Construction: Exploring structure learning integrated with model training and optimizing adjacency matrices.
  • Theoretical Analysis: Providing a deeper theoretical understanding of why multi-topology GNNs outperform single-topology ones.
  • Broader Applicability: Investigating the applicability of the proposed method to other educational data mining tasks.
  • Pre-trained Models: Developing a pipeline for pre-trained models that can be fine-tuned for specific problem-solving scenarios.

Summary updated

Abstract

Student performance prediction has attracted increasing attention in the field of educational data mining, or more broadly, intelligent education or “AI + education”. Accurate performance prediction plays a significant role in solving the problem of a student dropping out, promoting personalized learning and improving teaching efficiency, etc. Traditional student performance prediction methods usually ignore the potential (underlying) relationship among students. In this paper, we use graph structure to reflect the students’ relationships and propose a novel pipeline for student performance prediction based on newly-developed multi-topology graph neural networks (termed MTGNN). In particular, we propose various ways for graph construction based on similarity learning using different distance metrics. Based on the multiple graphs of different topologies, we design an MTGNN module, as a key module in the pipeline, to deal with the semi-supervised node classification problem where each node represents a student (and the node label is the student’s performance, e.g., Pass/Fail/Withdrawal). An attention-based method is developed to produce the unified graph representation in MTGNN. The effectiveness of the proposed pipeline is verified in a case study, where a real-world educational dataset and several existing approaches are used for performance comparison. The experiment results show that, compared with some traditional machine learning methods and the vanilla graph convolutional network with only a single graph topology, our proposed pipeline works effectively and favorably in student performance prediction.
BibTeX
@inproceedings{li2022study,
  author = {Ming Li and Xiangru Wang and Yi Wang and Yuting Chen and Yixuan Chen},
  title = {{Study-GNN: A Novel Pipeline for Student Performance Prediction Based on Multi-Topology Graph Neural Networks}},
  booktitle = {Sustainability},
  year = {2022},
  doi = {10.3390/su14137965},
}
Figure