TL;DR
This paper introduces the AI-Agent School (AAS), a novel multi-agent simulation system designed to model complex educational dynamics with high fidelity. AAS utilizes a "Zero-Exp" strategy with a dual memory system (experience and knowledge, each with short-term and long-term components) to enable AI agents to autonomously evolve and learn, generating realistic behavioral data for educational research and development.
What Problem Does This Paper Address?
The paper addresses the limitations of current AI agents in simulating educational settings. Existing agents often struggle with systematic modeling of teaching processes and accurately portraying the nuanced interactions of diverse participants (teachers and students). This fragmentation hinders the development of realistic educational simulations, which are crucial for advancing adaptive learning, optimizing teaching models, and creating effective educational tools like digital twins and training platforms. The authors aim to bridge this gap by creating a system that can capture complex, multi-participant dynamics and foster agent evolution for more realistic simulations.
Methods
The core of the proposed system is the AI-Agent School (AAS), a multi-agent simulation environment. Key methodological contributions include:
- Zero-Exp Strategy: This strategy enables agents to evolve from a "zero-experience" state to expert-level behavior using minimal initialization. It's driven by an "experience-reflection-optimization" cycle.
- Dual Memory System: Agents are equipped with a multi-layered memory system comprising:
- Working Memory: Holds immediate, context-window relevant information.
- Dual Memory Base: * Experience Base: Stores records of past events and interactions.
- Knowledge Base: Stores structured information (role-specific knowledge, general facts).
- Hierarchical Memory: Both Experience and Knowledge bases are further divided into:
- Long-term Memory: A comprehensive repository of all accumulated memories.
- Short-term Memory: A subset of salient memories deemed most relevant for current tasks, mimicking human focus.
- Self-Evolving Mechanism: Agents iteratively update their memory bases through preset behaviors and interaction data, leading to autonomous evolution and role-consistent behavior.
- Environment and Agent Design: The AAS environment is built using Tiled and Cocos, offering 25 distinct areas. Agent roles (teachers and students) are generated using LLMs (QwQ-32B) to create rich backgrounds and personality traits. Actions are categorized by role.
Datasets
The dataset used for training and evaluation was constructed through a multi-step process:
- LLM Generation: QwQ-32B generated initial role settings for 10 teacher and 40 student agents.
- Expert Refinement: Human experts discussed and modified roles, while the environment map and a 5-day course schedule were developed.
- Initial Simulation Data: Gemini-2.5-Pro generated an initial 5-day sequence of agent actions and interactions (movements, dialogues, activity performance), with classroom teaching data sourced from real classrooms.
- Final Dataset: This data was rigorously modified, reorganized, and reordered by educational experts to create the "Standard Group" dataset (ID 0), serving as ground truth for evaluation.
Key Insights and Ideas
- The Power of Dual Memory: Separating experience from knowledge, and managing them hierarchically (short-term vs. long-term), significantly enhances an agent's ability to learn, reflect, and generate relevant responses. This dual structure is more effective than unified memory or context-only approaches.
- Autonomous Evolution is Key: The "experience-reflection-optimization" cycle, powered by the memory system, allows agents to dynamically adapt and improve their behavior over time, moving beyond static role-playing.
- Bridging the "Era of Experience" to the "Era of Simulation": AAS aims to generate high-fidelity behavioral and interaction data, providing a foundation for a new era of simulation-driven educational research and development.
- LLMs as Building Blocks, Not Endpoints: While LLMs are used for role generation and agent responses, the AAS framework provides the structure for these agents to evolve and learn in a complex, dynamic environment, going beyond simple LLM prompting.
Key Results
- Automated Evaluation (ROUGE-L Scores): The full model (ID 1), incorporating both the dual KB/EB structure and the ST/LT hierarchy, consistently achieved the highest ROUGE-L scores across three different LLM models (GPT-40, Qwen3-235B-A22B, Qwen3-8B), indicating superior performance in generating responses aligned with the ground truth.
- Human Evaluation: Educational experts rated the full model (ID 1) as significantly more realistic than baselines lacking the full memory structure (e.g., Context Only, Unified Memory). The full model's perceived realism approached that of the expert-curated ground truth (ID 0) in later stages of the simulation.
- Contribution of Memory Components: Experiments showed that external memory is fundamental, the dual KB/EB structure offers advantages over unified memory, and the ST/LT hierarchy significantly improves performance by prioritizing salient memories.
Limitations and Future Work
- Simulation Scale: The current simulation is limited to 50 agents over 5 days. Scaling to a full school environment with hundreds or thousands of agents and longer durations presents significant computational and design challenges.
- LLM-based Cognition: Agents currently lack visual perception (no Vision-Language Models), limiting realism in interactions relying on non-verbal cues.
- Dataset Fidelity: The simulation's fidelity is dependent on the quality and diversity of the initial expert-curated dataset. Nuances of real-world cognition and social interaction are difficult to fully capture.
- LLM Biases: Agent behaviors are inherently constrained by the capabilities and potential biases of the underlying LLMs.
Future Work: * Scaling the AAS environment to larger agent populations and longer durations. * Incorporating Vision-Language Models (VLMs) or other multimodal models for enhanced perception. * Exploring alternative or hybrid LLM architectures. * Developing more sophisticated reflection and optimization mechanisms within the Zero-Exp framework. * Applying the generated high-fidelity data to specific educational applications (personalized learning, automated feedback). * Extending the AAS framework to other domains requiring complex experience accumulation and utilization.
Summary updated