Introduction
Intelligent Tutoring Systems (ITS) are designed to maintain student engagement, yet many current implementations are reactive in a delayed manner, adjusting lessons only after analyzing session data. Furthermore, much of the existing research on affective computing in education focuses on adults in controlled laboratory settings. This research addresses these gaps by exploring whether a child's imminent behavior can be predicted in real-time based on their facial expressions, specifically within the context of RoboTutor, an open-source Android tablet tutor for literacy and numeracy.
The study focuses on children aged 6-12, many of whom have limited prior schooling or technology access. The goal is to move beyond general engagement metrics to predict specific actions, such as prematurely exiting an activity, allowing for more timely and effective pedagogical interventions.
Methodology
The researchers utilized data from RoboTutor sessions to map facial expressions to pedagogical behaviors:
- Affective State Detection: The OpenFace toolkit extracted Facial Action Units (FAUs) from the tablet's front-facing camera.
- Emotion Mapping: These FAUs were mapped to six pedagogically relevant emotions: boredom, confusion, frustration, surprise, delight, and neutral.
- Data Integration: Detected emotions were synchronized with application log files recording user actions (e.g., button presses, activity completion, and answer correctness).
- Analysis: Statistical correlation and visualization identified links between these emotional states and subsequent student behaviors.
The dataset comprised 17 unique session pairs ($N=17$), with sessions lasting 20 to 30 minutes. Activities included racing games, writing exercises, and read-along stories.
Key Insights
Delight as a Predictor of Completion
A significant finding was the relationship between expressed delight and task persistence. Students who successfully completed activities exhibited significantly higher median levels of delight compared to those who did not ($p < 0.05$). This suggests that positive affective reinforcement is a strong indicator of a student's likelihood to finish a task.
The "Flow" of Neutrality in Performance
In the "bubble pop" activity, researchers observed a counterintuitive trend regarding performance. While one might expect positive emotions to correlate with success, the only state that positively correlated with correctness was "neutral." Every other emotion—including delight—showed a negative correlation with performance. This indicates that deep cognitive processing and "flow" often manifest as a neutral facial expression, whereas high-arousal emotions (even positive ones) may interfere with the focus required for certain tasks.
Surprise as a "Quit" Signal
While boredom was a frequent state, it was not the primary predictor of quitting. Instead, "surprise" emerged as a statistically significant indicator that a student was about to hit the "back" button. This suggests that a sudden mismatch in expectations—perhaps due to a difficult task or confusing interface—triggers a child to exit an activity prematurely.
Conclusion and Future Work
The study demonstrates that specific affective states are statistically significant markers for imminent student behavior. High delight predicts completion, while surprise often precedes quitting. Notably, a neutral expression is the best correlate for correctness in focused tasks.
The primary limitation of this work is the small sample size ($N=17$). Future work involves integrating these real-time predictive capabilities into RoboTutor’s intervention policy. By adjusting the tutor's strategy the moment a "quit" behavior is predicted, the system could significantly reduce session dropout rates and improve learning outcomes.
Summary updated