Abstract
Predicting student dropout in K-12 online tutoring presents unique challenges compared to traditional Massive Open Online Courses (MOOCs), including lower dropout rates, high interactivity across multiple modalities, and significant time sensitivity. This paper proposes a machine learning framework utilizing a Gradient Boosting Decision Tree (GBDT) model to identify at-risk students. By integrating multimodal in-class features with out-of-class activity logs and implementing a novel time-aware data augmentation strategy, the system achieves a 70% detection rate in a real-world production environment.
Introduction
While dropout prediction is well-studied in MOOCs, the K-12 online tutoring landscape is distinct. MOOCs typically involve adult learners with high attrition rates (70-93%). In contrast, K-12 tutoring is often a paid service with dropout rates below 20%. The authors identify four primary challenges in this domain: multimodal interactions (video, voice, text), high variability in student learning history length, time sensitivity (the "recency effect"), and severe data imbalance.
Methodology
The proposed framework centers on a GBDT model, leveraging a comprehensive feature set and a specialized data augmentation technique.
Multimodal Feature Engineering
The model incorporates three distinct categories of features to capture a holistic view of student engagement:
- In-class Features: Derived from classroom recordings, including prosodic (loudness, pitch), linguistic (speech speed, sentence length), and interaction metrics (Q&A frequency).
- Out-of-class Features: Service-side data such as pre-class negotiations, discount ratios, and post-class rescheduling frequency.
- Time-variant Features: Statistics calculated over specific lookback windows (e.g., activity in the last 7 days) to capture temporal trends.
Time-Aware Data Augmentation
To address data imbalance, the researchers introduced "pseudo-positive" samples. This approach assumes that students who eventually drop out exhibit "at-risk" behaviors in the preceding days. These samples are weighted using a convex function, ensuring the model prioritizes behaviors immediately prior to dropout, effectively modeling the "recency effect."
Results and Discussion
The study utilized a dataset of 3,922 students from a K-12 online tutoring platform, covering over 338,000 observation pairs with a 16.16% dropout rate.
Model Performance
Offline evaluations demonstrated that the proposed GBDT model consistently outperformed baseline methods like Logistic Regression (LR), Random Forest (RF), and Decision Trees (DT). Notably, the model maintained superior performance across various prediction horizons, ranging from 1 to 14 days ahead.
Key Insights
- The Recency Effect: Emphasizing recent behavior through convex weighting significantly improved predictive accuracy compared to linear or concave models.
- Feature Importance: Surprisingly, out-of-class service features (e.g., rescheduling frequency) were more predictive than in-class multimodal features, suggesting that administrative friction is a major driver of attrition in paid K-12 contexts.
- Prediction Horizon: Model performance is highest for short-term predictions (AUC 0.8262 for 1-day ahead) and gradually declines as the window extends to 14 days (AUC 0.7430), reflecting the volatility of student intent.
Conclusion
The framework's deployment in a live production environment resulted in the successful identification of over 70% of students who eventually dropped out, enabling timely interventions. Future work will explore Deep Neural Networks to capture more complex non-linear relationships and aim to improve ASR accuracy to refine linguistic feature extraction.
Summary updated