Meta Transfer Learning for Early Success Prediction in MOOCs

Vinitra Swamy, Mirko Marras, Tanja Käser

Abstract

Despite the increasing popularity of massive open online courses (MOOCs), many suffer from high dropout and low success rates. Early prediction of student success for targeted intervention is therefore essential to ensure no student is left behind in a course. There exists a large body of research in success prediction for MOOCs, focusing mainly on training models from scratch for individual courses. This setting is impractical in early success prediction as the performance of a student is only known at the end of the course. In this paper, we aim to create early success prediction models that can be transferred between MOOCs from different domains and topics. To do so, we present three novel strategies for transfer: 1) pre-training a model on a large set of diverse courses, 2) leveraging the pre-trained model by including meta information about courses, and 3) fine-tuning the model on previous course iterations. Our experiments on 26 MOOCs with over 145,000 combined enrollments and millions of interactions show that models combining interaction data and course information have comparable or better performance than models which have access to previous iterations of the course. With these models, we aim to effectively enable educators to warm-start their predictions for new and ongoing courses.

Introduction

Massive Open Online Courses (MOOCs) have democratized education, yet they face a persistent challenge: high dropout rates and low completion percentages. Identifying students at risk of failure early in a course is critical for timely intervention. However, traditional machine learning approaches often suffer from the "cold start" problem—they require historical data from the specific course being analyzed. This makes it impossible to predict success for new courses or to provide "live" predictions during a course's first iteration.

This research addresses the challenge of transferability. The goal is to build models that learn generalizable patterns of student success across diverse subjects and apply them to new, unseen contexts.

The Prediction Pipeline

The researchers developed a comprehensive pipeline to transform raw student interactions into actionable success predictions. This process involves four main stages: data collection, preprocessing, feature extraction, and classification.

Overview of the end-to-end prediction pipeline from data collection to classification.

The pipeline processes fine-grained "clickstream" events (e.g., video play/pause, quiz submissions) and course metadata. To simulate early prediction, the data is truncated at specific intervals, such as 40% or 60% of the course duration.

Model Architectures

Three deep learning architectures based on Bidirectional Long Short-Term Memory (BiLSTM) networks were proposed to handle the temporal nature of student behavior:

  1. Behavior Only (BO): A baseline model using 45 hand-crafted behavioral features, such as video interaction patterns and study regularity.
  2. Behavior + Time-wise Meta (BTM): This model injects course metadata (e.g., title, difficulty) into every weekly activity log, providing constant context to the behavioral sequence.
  3. Behavior + Static Meta (BSM): The most advanced approach, utilizing Attention Mechanisms to weigh relevant course characteristics. It processes behavior and metadata in separate branches before merging them via a projection layer for final prediction.
Architectural diagrams for the BO, BTM, and BSM models.

Course metadata, including text-based descriptions, was converted into mathematical vectors using FastText embeddings, allowing the models to "understand" the syllabus context.

Experimental Setup and Results

The study utilized a massive dataset of 26 MOOCs with over 145,000 enrollments. These courses spanned multiple languages, academic levels, and topics.

Key Findings

  • Transfer Performance: The best transfer models achieved a Balanced Accuracy (BAC) of approximately 70% on entirely new courses.
  • The Metadata Advantage: Incorporating course metadata improved performance by an average of 11.45% compared to simple behavior-only transfer. This suggests that metadata acts as a "bridge," helping the model contextualize behavior based on course difficulty and topic.
  • Early Signal Strength: High accuracy was achieved using only 40% of the course duration, offering a significant window for instructor intervention.
  • Context vs. History: Models using metadata often outperformed those trained on previous iterations of the same course, highlighting the power of cross-course generalization.

Conclusion

By leveraging meta transfer learning, this framework enables "warm-starting" success predictions for new MOOCs. The integration of course metadata with behavioral sequences allows models to generalize across domains, providing educators with a powerful tool to support students from the very beginning of their learning journey.

Summary updated

Abstract

Despite the increasing popularity of massive open online courses (MOOCs), many suffer from high dropout and low success rates. Early prediction of student success for targeted intervention is therefore essential to ensure no student is left behind in a course. There exists a large body of research in success prediction for MOOCs, focusing mainly on training models from scratch for individual courses. This setting is impractical in early success prediction as the performance of a student is only known at the end of the course. In this paper, we aim to create early success prediction models that can be transferred between MOOCs from different domains and topics. To do so, we present three novel strategies for transfer: 1) pre-training a model on a large set of diverse courses, 2) leveraging the pre-trained model by including meta information about courses, and 3) fine-tuning the model on previous course iterations. Our experiments on 26 MOOCs with over 145,000 combined enrollments and millions of interactions show that models combining interaction data and course information have comparable or better performance than models which have access to previous iterations of the course. With these models, we aim to effectively enable educators to warm-start their predictions for new and ongoing courses.
BibTeX
@inproceedings{swamy2022meta,
  author = {Vinitra Swamy and Mirko Marras and Tanja Käser},
  title = {{Meta Transfer Learning for Early Success Prediction in MOOCs}},
  booktitle = {L@S},
  year = {2022},
  doi = {10.1145/3491140.3528273},
}
Figure