ADVISOR: A Machine Learning Architecture for Intelligent Tutor Construction

AAAI 2000

TL;DR

This paper introduces ADVISOR, a novel two-agent machine learning architecture for building intelligent tutoring systems (ITS). ADVISOR centralizes ITS reasoning, allowing for customizable teaching goals and simplified ITS improvement by learning student behavior and deriving optimal teaching policies through reinforcement learning.

What Problem Does This Paper Address?

Building effective intelligent tutoring systems (ITS) is challenging due to their high development cost and lack of configurability once deployed. Traditional ITS often rely on fixed teaching strategies derived from expert knowledge, which can be difficult to adapt to specific educational objectives or new contexts. Furthermore, the reasoning behind an ITS's teaching decisions can be opaque and hard to evaluate. This paper addresses the need for more adaptable, configurable, and transparent ITS by proposing a machine learning-driven architecture that can learn teaching policies from data.

Methods

ADVISOR employs a two-agent architecture:

  • Population Student Model (PSM): This agent learns a model of how students, as a population, behave within the tutor. It uses data from previous student interactions to predict two key metrics for a given context: the probability of a correct response and the time a student will take to respond. The PSM uses linear regression models trained on features describing the student, topic, problem, and context. Crucially, the PSM's predictions are designed to be independent of the specific teaching policy being used.
  • Pedagogical Agent (PA): This agent is responsible for learning a teaching policy to achieve a predefined educational goal. It uses reinforcement learning (specifically TD(0) with state-value learning) and is provided with the PSM (acting as a simulated environment) and a high-level teaching goal (e.g., "Students should make precisely 1 mistake per problem"). The PA experiments with different teaching actions (topic selection, problem generation, feedback) within the simulation to discover a policy that optimizes the given goal.

Datasets

The PSM was trained on approximately 11,000 training instances collected from previous deployments of the AnimalWatch tutor with rural fourth and fifth-grade students. The evaluation study involved an urban, sixth-grade class.

Key Insights and Ideas

  • Decoupling Student Model from Teaching Policy: A critical insight is that the student model (PSM) should predict student behavior independently of the current teaching policy. This allows the PSM to be trained on data from tutors with different policies and to remain useful even when the PA changes the tutor's teaching strategy.
  • Reinforcement Learning for Teaching Policy: Using reinforcement learning for the PA allows the system to learn optimal teaching strategies directly from data and a defined objective, rather than relying solely on pre-programmed heuristics. This makes the system adaptable to new goals.
  • Centralized Reasoning for Simplicity and Improvement: By centralizing the ITS's reasoning into the ADVISOR architecture, improvements to the PSM or PA directly benefit the entire tutoring system, simplifying maintenance and enhancement.
  • Data-Driven ITS Construction: The paper highlights the growing feasibility of constructing ITS using data-driven techniques, moving away from purely expert-driven knowledge engineering.

Key Results

  • PSM Accuracy: The PSM achieved a correlation of 0.629 (R² ≈ 0.4) for predicting student response time and a correlation of 0.484 (R² ≈ 0.23) for predicting response correctness. While correctness prediction accuracy was lower for individual cases, it was deemed sufficient for simulation purposes.
  • PA Performance: In an evaluation with actual students, the ADVISOR-driven AnimalWatch tutor reduced the average time students spent per problem from 39.7 seconds (classic tutor) to 27.7 seconds, a statistically significant improvement of 30%. This demonstrated that the PA successfully learned a teaching policy to meet the goal of minimizing problem-solving time.
  • Topic Area Speedup: Students using ADVISOR solved whole number and fraction problems significantly faster than those using the classic tutor, even when controlling for cognitive ability.

Limitations and Future Work

  • Template Restrictions: The current implementation faced limitations due to a finite number of problem templates, leading to the tutor sometimes selecting less optimal templates or difficulty levels to avoid repetition. This is a data limitation that can be addressed by adding more templates.
  • Training Data Bias: The PSM's training data was biased towards students who were "ready" for the topics presented. This makes it challenging to predict student behavior when they are presented with unprepared-for material. The authors compromised by filtering topics, but acknowledge this as a fundamental AI architecture challenge.
  • Online Learning: The authors plan to investigate online, real-time learning for both the PSM and PA to enhance adaptivity. This involves integrating individual student actions into the PSM and developing methods for updating the teaching policy in real-time without interfering with tutor response times.
  • More Complex Goals: Future work includes enabling ADVISOR to handle more complex and delayed teaching goals, such as mastering an entire topic rather than just optimizing per-problem time. This will require scaling up the simulation and learning mechanisms.

Summary updated

BibTeX
@inproceedings{2000advisor,
  title = {{ADVISOR: A Machine Learning Architecture for Intelligent Tutor Construction}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2000},
}

Loading related content...

Figure