Learning Latent Engagement Patterns of Students in Online Courses

Arti Ramesh, Dan Goldwasser, Bert Huang, Hal Daumé Iii, Lise Getoor

Introduction: The MOOC Attrition Challenge

Massive Open Online Courses (MOOCs) have revolutionized access to higher education, yet they face a persistent hurdle: extremely high attrition rates. Often, fewer than 10% of registered students complete their courses. While digital platforms provide a wealth of clickstream data, these raw metrics—video views, forum posts, and quiz attempts—frequently fail to capture the underlying psychological state of "engagement."

Researchers from the University of Maryland and UC Santa Cruz propose a novel framework using Probabilistic Soft Logic (PSL) to model engagement as a latent variable. By distinguishing between active and passive participation, their approach identifies at-risk students much earlier and more accurately than traditional "flat" feature models.

Dataset and Problem Scale

The study analyzed three Coursera MOOCs: Surviving Disruptive Technologies (DISR-TECH), Women and the Civil Rights Movement (WOMEN-CIVIL), and Gene and the Human Condition (GENE).

Student activity levels across three MOOC datasets, highlighting the sharp drop-off between registration and completion.

As shown in the data, there is a massive disparity between the number of students who register and those who "survive" to the end. For instance, in the DISR-TECH course, while over 15,000 students registered, only a tiny fraction completed the course. The challenge lies in predicting which of the thousands of "passive" students (those viewing lectures but not posting) will eventually drop out.

Methodology: Probabilistic Soft Logic

The researchers utilized Probabilistic Soft Logic (PSL), a declarative language for collective probabilistic reasoning. Unlike binary logic, PSL operates on "soft" truth values between 0 and 1, allowing it to handle the inherent uncertainty in student behavior.

The core innovation is the PSL-LATENT model. While a standard model (PSL-DIRECT) maps observable actions directly to survival, the LATENT model introduces intermediate hidden states:

  • Active Engagement: Derived from forum posts and quiz submissions.
  • Passive Engagement: Derived from lecture views and forum "lurking" (voting/subscribing).

By modeling these latent states, the system can reason about the type of student engagement rather than just the volume of clicks.

Key Results and Feature Analysis

The LATENT model consistently outperformed baselines across all courses. A critical finding was the effectiveness of early-stage prediction. Even using only data from the "start" phase of a course, the model achieved significant predictive accuracy.

Performance analysis of different feature sets across course phases in the GENE course.

As illustrated in the feature analysis for the GENE course, combining all features leads to the highest performance, particularly when looking at the "start-end" aggregate. Interestingly, the "middle" phase of the course emerged as a pivotal window; students whose engagement flags during this period are highly likely to become dropouts.

Insights for Educators

  1. Passive Engagement Matters: "Silent" students who watch lectures but don't post are not necessarily disengaged. The latent model successfully identifies those likely to finish based on passive cues.
  2. Lecture Viewing is a Primary Signal: Across all datasets, the number of lectures viewed remains the strongest individual predictor of student success.
  3. Early Intervention is Possible: The high predictive scores in the "start" phase suggest that instructors can identify at-risk students within the first week or two, allowing for targeted outreach.

Conclusion

By moving beyond simple clickstream tracking to modeling the latent patterns of engagement, this research provides a more nuanced understanding of student behavior in MOOCs. The PSL-LATENT framework offers a powerful tool for educators to improve retention through early, data-driven intervention.

Summary updated

Abstract

Maintaining and cultivating student engagement is critical for learning. Understanding factors affecting student engagement will help in designing better courses and improving student retention. The large number of participants in massive open online courses (MOOCs) and data collected from their interaction with the MOOC open up avenues for studying student engagement at scale. In this work, we develop a framework for modeling and understanding student engagement in online courses based on student behavioral cues. Our first contribution is the abstraction of student engagement types using latent representations. We use that abstraction in a probabilistic model to connect student behavior with course completion. We demonstrate that the latent formulation for engagement helps in predicting student survival across three MOOCs. Next, in order to initiate better instructor interventions, we need to be able to predict student survival early in the course. We demonstrate that we can predict student survival early in the course reliably using the latent model. Finally, we perform a closer quantitative analysis of user interaction with the MOOC and identify student activities that are good indicators for survival at different points in the course.
BibTeX
@inproceedings{ramesh2014learning,
  author = {Arti Ramesh and Dan Goldwasser and Bert Huang and Hal Daumé III and Lise Getoor},
  title = {{Learning Latent Engagement Patterns of Students in Online Courses}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2014},
  doi = {10.1609/aaai.v28i1.8920},
}
Figure