MathVC: An LLM-Simulated Multi-Character Virtual Classroom for Mathematics Education

Murong Yue, Wijdane Mifdal, Yixuan Zhang, Jennifer Suh, Ziyu Yao

Abstract

Mathematical modeling (MM) is considered a fundamental skill for students in STEM disciplines. Practicing the MM skill is often the most effective when students can engage in group discussion and collaborative problem-solving. However, due to unevenly distributed teachers and educational resources needed to monitor such group activities, students do not always receive equal opportunities for this practice. Excitingly, large language models (LLMs) have recently demonstrated strong capability in both modeling mathematical problems and simulating characters with different traits and properties. Drawing inspiration from the advancement of LLMs, in this work, we present MATHVC, the very first LLM-powered virtual classroom containing multiple LLM-simulated student characters, with whom a human student can practice their MM skill. To encourage each LLM character's behaviors to be aligned with their specified math-relevant properties (termed "characteristics alignment") and the overall conversational procedure to be close to an authentic student MM discussion (termed "conversational procedural alignment"), we proposed three innovations: integrating MM domain knowledge into the simulation, defining a symbolic schema as the ground for character simulation, and designing a meta planner at the platform level to drive the conversational procedure. Through experiments and ablation studies, we confirmed the effectiveness of our simulation approach and showed the promise for MATHVC to benefit real-life students in the future.

Introduction

Collaborative Problem Solving (CPS) is essential for deep mathematical learning—students learn best when they have to explain their logic, critique others, and negotiate solutions. However, real-world classrooms face significant hurdles: large class sizes make it impossible for teachers to monitor every group, and social dynamics often lead to "high-competency" students dominating the conversation while others disengage.

The MathVC interface featuring virtual peers Alice and Bob interacting with a human student.

MathVC addresses the lack of high-quality, available peers by providing a multi-persona virtual classroom where Large Language Models (LLMs) simulate "virtual peers." Unlike standard AI assistants designed for immediate correctness, MathVC peers are engineered to be fallible and follow a structured multi-stage discussion process, creating a psychologically safe environment for students to practice "math talk."

System Architecture and Methodology

To move beyond simple prompting and achieve both "persona alignment" (consistent character traits) and "procedural alignment" (authentic discussion flow), MathVC utilizes a modular architecture driven by symbolic constraints.

The MathVC architecture, illustrating the initialization of task schemas and the persona simulation stack.

Meta-Planning Module

This module acts as the classroom's "orchestrator." It includes a Collaboration Stage Monitor that tracks the discussion through five stages (Sense-making, Team Organization, Planning, Execution, and Validation) based on the PISA 2015 framework. It also features a Next Speaker Control to manage turn-taking and ensure the human student has ample opportunity to participate.

Persona Simulation Stack

To prevent the LLM from defaulting to a "perfect assistant" mode, the authors used Symbolic Schemas to anchor the AI's knowledge state:

  • Task Schema: A ground-truth structural representation of the math problem.
  • Persona Schema: A version of the task schema injected with specific, teacher-defined misconceptions or errors.
  • Schema Modifier: Dynamically updates the agent's internal state as the conversation progresses (e.g., if the human student corrects the agent, the agent "learns" and updates its schema).

Dialogue Act & Response Generation

Instead of letting the LLM wander, the system selects a specific "intent" (e.g., "Ask a clarifying question") before generating a response, ensuring the agents stay focused and concise.

Experimental Results

The system, powered by GPT-4o, was evaluated using 9 grade-appropriate math problems (Grades 6-8). A study involving 14 middle schoolers yielded 40 dialogue sessions (695 total messages).

Key Insights

  • The Value of Fallibility: Students found the agents more authentic when they made mistakes. An agent that messes up a calculation and needs to be corrected creates a "learning by teaching" dynamic.
  • Heterogeneity as a Scaffold: By providing two distinct personas—"Alice" (limited skill) and "Bob" (advanced skill)—the system forced students to navigate conflicting perspectives, encouraging more robust arguments.
  • Psychological Safety: Students felt more comfortable making mistakes in front of AI peers than they might in a real classroom, viewing errors as diagnostic tools rather than failures.

Conclusion and Future Work

MathVC demonstrates that symbolic schemas are necessary to prevent "persona drift" toward correctness in LLMs. While students reported high levels of constructive interaction (Mean = 4.36/5), they noted that agents occasionally sounded too formal. Future work will focus on improving linguistic authenticity and exploring "fading" mechanisms to gradually reduce AI support as students become more proficient.

Summary updated

Abstract

Mathematical modeling (MM) is considered a fundamental skill for students in STEM disciplines. Practicing the MM skill is often the most effective when students can engage in group discussion and collaborative problem-solving. However, due to unevenly distributed teachers and educational resources needed to monitor such group activities, students do not always receive equal opportunities for this practice. Excitingly, large language models (LLMs) have recently demonstrated strong capability in both modeling mathematical problems and simulating characters with different traits and properties. Drawing inspiration from the advancement of LLMs, in this work, we present MATHVC, the very first LLM-powered virtual classroom containing multiple LLM-simulated student characters, with whom a human student can practice their MM skill. To encourage each LLM character's behaviors to be aligned with their specified math-relevant properties (termed "characteristics alignment") and the overall conversational procedure to be close to an authentic student MM discussion (termed "conversational procedural alignment"), we proposed three innovations: integrating MM domain knowledge into the simulation, defining a symbolic schema as the ground for character simulation, and designing a meta planner at the platform level to drive the conversational procedure. Through experiments and ablation studies, we confirmed the effectiveness of our simulation approach and showed the promise for MATHVC to benefit real-life students in the future.
BibTeX
@misc{yue2024mathvc,
  author = {Murong Yue and Wijdane Mifdal and Yixuan Zhang and Jennifer Suh and Ziyu Yao},
  title = {{MathVC: An LLM-Simulated Multi-Character Virtual Classroom for Mathematics Education}},
  year = {2024},
  eprint = {2404.06711},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2404.06711},
}
Figure