Abstract
Massive Open Online Courses (MOOCs) have shown rapid development in recent years, allowing learners to access high-quality digital material. Because of facilitated learning and the flexibility of the teaching environment, the number of participants is rapidly growing. However, extensive research reports that the high attrition rate and low completion rate are major concerns. In this paper, the early identification of students who are at risk of withdrawal and failure is provided. Therefore, two models are constructed: the at-risk student model and the learning achievement model. These models have the potential to detect students who are in danger of failing or withdrawing at an early stage of the online course. The results reveal that all classifiers gain good accuracy across both models, with the highest performance yielded by Gradient Boosting Machines (GBM) with values of 0.894 and 0.952 for the first and second models, respectively. The proposed frameworks can be used to assist instructors in delivering intensive intervention support to at-risk students.
Introduction
While Massive Open Online Courses (MOOCs) have democratized education by providing global access to elite university content, they face a persistent challenge: notoriously high attrition rates. Most students who enroll in a MOOC never reach the finish line. Identifying these "at-risk" students early is difficult in a virtual environment where instructors cannot rely on physical cues or face-to-face interaction to gauge engagement.
This research addresses the critical need for early intervention. By predicting which students are likely to drop out or fail within the first week of a course, or based on their historical behavior, educational institutions can provide targeted support to improve retention. The core challenge lies in the "noisy" nature of online clickstream data and the significant class imbalance between the majority of students who fail or withdraw and the minority who successfully complete the course.
Methodology
The researchers developed a dual-framework approach to capture different facets of student risk:
1. At-Risk Student Model
This model focuses on "motivation trajectories." Rather than viewing engagement as a static snapshot, the model categorizes students into groups—Intrinsic, Extrinsic, or Amotivated—based on their interactions across multiple course intervals. It specifically tracks how motivation shifts over time, identifying "amotivation" when a student drops out within the first week of a new course despite previous engagement.
2. Learning Achievement Model
This framework predicts final academic outcomes (Pass/Fail) based on granular behavioral features. These include clickstream data such as forum posts, video views, homepage visits, and quiz attempts.
Data Pre-processing and Feature Engineering
To ensure robust model performance, several sophisticated pre-processing steps were employed:
- SMOTE (Synthetic Minority Over-sampling Technique): To address the dataset imbalance (approximately 78% failure vs. 22% success), SMOTE was used to generate synthetic examples of the minority class, preventing model bias toward the majority class.
- Feature Selection: Chi-square tests were utilized to filter out irrelevant variables, focusing the models on high-impact features. Notably, the "number of days active" during the second interval of a course emerged as the strongest predictor of continued enrollment.
- Algorithms: The study compared five classifiers: Random Forest (RF), Feedforward Neural Networks (NNET1), Multi-Layer Perceptron (NNET2), Gradient Boosting Machines (GBM), and Generalized Linear Models (GLM).
Datasets
The study utilized two major benchmarks in the field of learning analytics:
- Harvard/MIT Dataset: Contains clickstream data from four diverse courses, including Computer Science and Health, providing a view of interactions with digital courseware.
- OULAD (Open University Learning Analytics Dataset): A robust longitudinal dataset from the UK’s Open University containing demographic data and Virtual Learning Environment (VLE) interactions for Social Science courses.
Results and Discussion
The Gradient Boosting Machine (GBM) emerged as the superior algorithm across both frameworks:
- Dropout Prediction: GBM achieved an accuracy of 0.894.
- Achievement Prediction: Both GBM and NNET1 reached an impressive accuracy of 0.952.
While Neural Networks performed consistently well, Random Forest (RF) yielded the lowest accuracy in the at-risk framework (0.866). This suggests that boosting and deep learning architectures are better equipped to handle the non-linear, complex nature of student behavior data. Furthermore, the study found that reducing the number of variables through feature selection actually improved performance and reduced overfitting, proving that a few high-quality behavioral metrics are more valuable than a high volume of noisy data.
Conclusion and Future Work
The proposed models demonstrate that motivation is a trajectory, and early clickstream patterns are highly reliable predictors of academic success. However, limitations remain regarding data granularity; the Harvard dataset provided summary totals rather than timestamped sequences, which limits the tracking of micro-behaviors.
Future research will explore the use of Deep Learning, specifically Convolutional Neural Networks (CNNs), to automatically extract features from raw student records. This could potentially eliminate the need for manual feature engineering and allow for even earlier, more automated interventions to save MOOC completion rates.
Summary updated