Personalized Robot Tutoring Using the Assistive Tutor POMDP (AT-POMDP)

Aditi Ramachandran, Sarah Strohkorb Sebo, Brian ScassellatiAAAI 2019 · DOI

Abstract

One-on-one human tutoring is highly effective but difficult to scale. Social robots offer a potential solution, yet most current systems lack the ability to adapt to a student's emotional and cognitive state simultaneously. This paper introduces the Assistive Tutor Partially Observable Markov Decision Process (AT-POMDP), a computational framework that allows robot tutors to personalize instruction by modeling both a student’s math mastery and their engagement level. In a field study involving fourth-grade students learning long division, those taught by a robot using the AT-POMDP logic showed significantly higher learning gains compared to a control group receiving a fixed sequence of help.

Introduction

Expert human tutors do more than just correct mistakes; they maintain a "working model" of a student's affective state. They intuitively know when to provide a step-by-step tutorial, when to offer a subtle hint, and when a student is disengaged and needs a break. Replicating this intuition in social robots requires a robust algorithm capable of handling the inherent uncertainty of a classroom environment, where a student's internal state cannot be directly observed.

A fourth-grade student interacting with the robot tutor during a math session.

The AT-POMDP Framework

The researchers formulated the tutoring challenge as an Assistive Tutor Partially Observable Markov Decision Process (AT-POMDP). Because a robot cannot directly "see" a student's true knowledge or engagement, it maintains a belief state—a probability distribution over possible states—based on observable behaviors such as answer accuracy and response speed.

State and Action Space

The model tracks three primary dimensions:

  1. Knowledge Level: Four discrete levels of mastery.
  2. Engagement: Categorized as Low or High.
  3. Attempt Number: Tracking progress within a specific problem.

To influence these states, the robot can choose from six distinct actions: interactive tutorials, worked examples, hints, "think-aloud" prompts, breaks (such as a quick game of Tic-Tac-Toe), or "no-action" to allow for independent work. The policy was optimized using a reward function that incentivizes gains in knowledge and engagement while accounting for the time cost of each intervention.

Experimental Evaluation

The model's parameters were initially derived from a prior dataset of robot-student interactions. To test the framework, the researchers conducted a study with 28 fourth-grade students over five sessions. Students were randomly assigned to either the AT-POMDP group or a "Fixed" policy group, which provided help in a standard progressive sequence (think-aloud → hint → example → tutorial).

Results and Discussion

The AT-POMDP approach yielded substantial improvements in student performance. Students in the AT-POMDP group achieved a normalized learning gain of 0.41, compared to just 0.08 in the Fixed group. This difference is both statistically significant and educationally substantial.

Quantitative results showing (a) average robot actions per session, (b) pre- and post-test scores, and (c) normalized learning gains.

Crucially, the gains were not the result of the robot providing more help, but rather smarter help. Both groups received a similar number of interventions (approximately 19 actions over five sessions). The AT-POMDP robot learned to identify "blind guessing" and responded with breaks to reset engagement, which led to a 40% accuracy boost on subsequent math attempts for disengaged students. Furthermore, the model frequently chose "no-action" for high-performing students, preventing unnecessary interruptions and fostering autonomy.

Conclusion

The AT-POMDP framework demonstrates that modeling engagement is as critical as modeling knowledge for effective automated tutoring. While the study focused on long division, the results suggest that cost-aware, affective-sensitive policies can significantly enhance the efficacy of social robots in educational settings. Future work will explore adapting the model to individual student preferences and scaling the framework to other subject areas.

Summary updated

BibTeX
@inproceedings{ramachandran2019personalized,
  author = {Aditi Ramachandran and Sarah Strohkorb Sebo and Brian Scassellati},
  title = {{Personalized Robot Tutoring Using the Assistive Tutor POMDP (AT-POMDP)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.1609/aaai.v33i01.33018050},
}
Figure