Ranking-Based At-Risk Student Prediction Using Federated Learning and Differential Features

Shunsuke Yoneda, Valdemar Svábenský, Gen Li, Daisuke Deguchi, Atsushi Shimada

Introduction

Educational Data Mining (EDM) has become a vital tool for improving student outcomes by identifying those at risk of academic failure before it occurs. By analyzing "learning logs"—digital footprints such as page turns, highlighting, and note-taking in e-textbooks—researchers can predict performance trends. However, two significant barriers prevent the widespread adoption of these models: the sensitive nature of student data, which is often protected by strict privacy regulations, and the "context gap," where student behavior varies so significantly between subjects that models trained on one course often fail in another.

This paper introduces a novel framework that addresses these challenges by combining Federated Learning (FL) with a Differential Features (DF) approach. This system allows for high-accuracy, privacy-preserving predictions that generalize across diverse academic environments.

The Problem: Privacy Silos and Contextual Variability

Currently, most high-performance machine learning models require data to be pooled into a single central server. In the educational sector, privacy laws often prevent schools or departments from sharing raw student logs. This creates "privacy silos" that limit the amount of data available for training.

Furthermore, traditional models rely on absolute behavior scores (e.g., "total pages read"). These metrics are highly context-dependent; a student reading 50 pages a week might be an overachiever in a literature course but a laggard in a fast-paced technical seminar. Because the underlying data distributions shift between courses, models trained on absolute values struggle to maintain accuracy when applied to new subjects.

Methodology

The researchers propose a two-pronged solution to bridge the privacy and context gaps:

1. Federated Learning (FL)

To maintain privacy, the authors utilize the FedAvg algorithm. In this setup, the data never leaves its local server (the "client"). Instead, the model is sent to the data. Each client trains the model on its local logs and sends only the resulting mathematical weights back to a central server. The server aggregates these weights to update a global model, ensuring that no raw student data is ever exposed.

2. Differential Features (DF) and Ranking

The "secret sauce" of this research is the shift from absolute to relative metrics. Instead of feeding the model a single student’s activity, the researchers calculate the difference between pairs of students.

By training the model on these pairwise differences, the system learns to predict a "difference score" rather than a grade. These scores are then used to create a risk ranking within a class. This approach is inherently more robust because it focuses on how a student deviates from their specific class average, making the model adaptable to different teaching styles and course lengths.

Experimental Setup

The study utilized a comprehensive dataset from Kyushu University, collected via the BookRoll e-textbook platform:

  • Training Data: 1,136 students across 12 different courses (e.g., Information and Communication, Fortran) over a four-year period.
  • Validation Data: 5 separate "hold-out" courses to test the model’s ability to generalize to entirely new academic contexts.
  • Feature Engineering: The team used E2Vec, a method that converts sequences of learning operations into 100-dimensional vectors, capturing the temporal rhythm of student engagement.

Key Results and Insights

Privacy Without Performance Loss

The Federated Learning model achieved performance levels nearly identical to "Centralized Learning" models. In terms of nDCG (Normalized Discounted Cumulative Gain), a standard measure for ranking quality, the FL model reached scores as high as 0.95 in several courses.

The Advantage of Differential Features

Models using Differential Features consistently outperformed those using standard absolute features. For instance, in one test course (B-2020), the DF approach improved the PR-AUC (Precision-Recall Area Under Curve) from 0.37 to 0.46. Additionally, the pairwise approach acts as a form of data augmentation; for a class of $n$ students, it creates $n(n-1)$ data points, which helps prevent overfitting in smaller classes.

Early Warning Capability

Crucially, the system proved effective as an early warning tool. By the midpoint of a semester, the model could identify at-risk students with high precision (Top-15 precision > 0.7 in most cases). This provides a sufficient window for instructors to provide meaningful interventions.

Conclusion and Future Work

This research demonstrates that it is possible to build highly effective predictive models for education without compromising student privacy. By focusing on relative student behavior through Differential Features, the model overcomes the traditional "context gap" that plagues absolute-value models.

While the results are promising, the authors note that the study was limited to a single institution and did not measure the communication overhead of the FL process. Future research will focus on improving the interpretability of the model—helping instructors understand why a student is at risk—and testing the framework across multiple universities.

The modeling code has been made available on GitHub to support further development in the field.

Summary updated

BibTeX
@inproceedings{yoneda2025ranking,
  author = {Shunsuke Yoneda and Valdemar Svábenský and Gen Li and Daisuke Deguchi and Atsushi Shimada},
  title = {{Ranking-Based At-Risk Student Prediction Using Federated Learning and Differential Features}},
  booktitle = {EDM},
  year = {2025},
}
Figure