Towards Goal-oriented Intelligent Tutoring Systems in Online Education

Yang Deng, Zifeng Ren, An Zhang, Tat-Seng Chua

TL;DR

This paper introduces Goal-oriented Intelligent Tutoring Systems (GITS), a new task focused on proactively guiding students toward mastering specific concepts. The authors propose PAI (Planning-Assessment-Interaction), a graph-based reinforcement learning framework that leverages cognitive structures and dynamic student modeling to deliver personalized, efficient learning paths.

What Problem Does This Paper Address?

Traditional Intelligent Tutoring Systems (ITS) are often reactive—they respond to student questions or provide feedback on specific exercises in isolation. While helpful, they lack a "big picture" strategy. They don't proactively plan a sequence of lessons to reach a long-term pedagogical goal, such as mastering "Pythagorean Theorem."

The authors identify a gap in proactive engagement: the ability of a system to decide what to teach next and when to assess a student's progress. A good tutor must balance two conflicting needs:

  1. Tutoring: Providing exercises that are neither too easy (boring) nor too hard (frustrating).
  2. Assessment: Checking for mastery without over-testing, which can exhaust a student's patience and lead them to quit.
Fig. 1. The workflow of GITS. The goal of the tutoring system is to educate the student a specific target concept through a multi-turn interaction session. Specifically, the planning of GITS involves two types of actions, including 1) Tutor an Exercise aims to tutor the student to comprehend an exercise for improving their understandings of the target concept. Afterwards, the system will decide the next action. 2) Assess the Concept aims to assess the student regarding their mastery of the target concept. If the student fails to pass the assessment, the system will decide the next action. The interaction terminates if the student masters the target concept or quits the interactive learning session.

Methods

The proposed framework, PAI, treats tutoring as a Markov Decision Process (MDP) optimized via Reinforcement Learning (RL). It consists of three core components:

  • Planning: PAI builds a unified Cognitive Graph that links students, exercises, and concepts, including prerequisite relationships (e.g., you must understand "Triangles" before "Right Triangles"). It uses a Graph Convolutional Network (GCN) to learn rich state representations and a prerequisite-guided strategy to narrow down the massive space of possible exercises to a manageable set of candidates.
  • Assessment: To train the system without needing thousands of hours of real human interaction, the authors use a User Simulator based on a cognitive diagnosis model called NeuralCD. Crucially, this simulator is dynamically updated during a session to reflect the student's learning progress in real-time.
  • Interaction: The core policy is driven by a Prioritized Dueling Double Deep Q-Network (PDDDQN). The agent receives rewards for successful mastery and small penalties for failed assessments or exercises that are too difficult, teaching it to be both effective and efficient.
Fig. 2. The overview of the Planning-Assessment-Interaction (PAI) framework. PAI consists of three main components. 1) The Planning component (§4.1) applies cognitive graph structure for state representation learning and prerequisite structure for action selection. 2) The Assessment component (§4.2) simulates the student rewards via dynamically updated cognitive diagnosis. 3) The Interaction component (§4.3) performs the reinforcement learning with a Deep Q-Network.

Datasets

The researchers constructed three benchmark datasets by extracting student-exercise records, concept relations, and prerequisites from the MOOCCubeX repository:

  • Computer Science: 1,966 users, 215 concepts.
  • Math: 2,701 users, 129 concepts.
  • Psychology: 4,710 users, 298 concepts.

These datasets provide a standardized way to test how well a GITS can handle different subjects with varying levels of complexity and prerequisite density.

Key Insights and Ideas

  • Prerequisites as a Search Filter: One of the biggest challenges in RL for education is the "action space"—there are thousands of exercises to choose from. By using the prerequisite graph to filter candidates, PAI significantly improves learning efficiency.
  • The Patience Constraint: The model explicitly accounts for "student patience." If a tutor repeatedly gives exercises that are too hard or tests the student too often, the simulated student "quits." This forces the RL agent to find the shortest path to mastery.
  • Dynamic vs. Static Modeling: Unlike traditional models that assume a student's knowledge is static during a session, PAI's simulator updates its parameters after every interaction, mimicking how a real student learns as they go.

Key Results

PAI consistently outperformed heuristic, supervised, and vanilla RL baselines across all three subjects:

  • Success Rate (SR): PAI achieved significantly higher success rates (e.g., 0.534 in Math vs. 0.491 for the best baseline).
  • Efficiency: It reached mastery in fewer turns. In the Computer Science dataset, PAI averaged 16.223 turns, outperforming all other methods.
  • Robustness: Qualitative analysis showed that while simpler methods (like Greedy or KNN) work for easy concepts, they fail on hard concepts where strategic planning is required. PAI proved much more robust for complex, "hard-to-learn" topics.

Limitations and Future Work

The authors acknowledge a few key limitations:

  • Simulation Reliance: Due to the high cost of human studies, the evaluation relies on simulators. While NeuralCD is a strong model, real-world student behavior is even more unpredictable.
  • Lack of Natural Language: The current system focuses on strategy (which exercise to pick) rather than content (generating the text of the explanation).
  • Future Directions: The authors suggest integrating Large Language Models (LLMs) to handle the dialogue generation side, creating a system that is both strategically smart and conversationally fluent. They also propose exploring hybrid policies that adapt based on the inherent difficulty of the target concept.

Summary updated

Abstract

Interactive Intelligent Tutoring Systems (ITSs) enhance traditional ITSs by promoting effective learning through interactions and problem resolution in online education. Yet, proactive engagement, prioritizing resource optimization with planning and assessment capabilities, is often overlooked in current ITS designs. In this work, we investigate a new task, named Goal-oriented Intelligent Tutoring Systems (GITS), which aims to enable the student's mastery of a designated concept by strategically planning a customized sequence of exercises and assessment. To address the problem of goal-oriented policy learning in GITS, we propose a novel graph-based reinforcement learning framework, named Planning-Assessment-Interaction (PAI). Specifically, we first leverage cognitive structure information to improve state representation learning and action selection for planning the next action, which can be either to tutor an exercise or to assess the target concept. Further, we use a dynamically updated cognitive diagnosis model to simulate student responses to exercises and concepts. Three benchmark datasets across different subjects are constructed for enabling offline academic research on GITS. Experimental results demonstrate the effectiveness and efficiency of PAI and extensive analyses of various types of students are conducted to showcase the challenges in this task.
BibTeX
@article{deng2023towards,
  author = {Yang Deng and Zifeng Ren and An Zhang and Tat-Seng Chua},
  title = {{Towards Goal-oriented Intelligent Tutoring Systems in Online Education}},
  journal = {ACM Trans. Information Systems},
  year = {2023},
}
Figure