TL;DR
This paper introduces a novel "Blackboard-based Dynamic Instructional Planner" (BB-IP) designed to create more flexible and adaptive intelligent tutoring systems (ITS). By leveraging a blackboard architecture, BB-IP can generate personalized lesson plans, dynamically revise them in response to student interactions and changing conditions, and integrate opportunistic teaching with structured lesson delivery.
What Problem Does This Paper Address?
Traditional Intelligent Tutoring Systems (ITS) often fall into two camps: those that follow pre-defined, static lesson plans (like traditional CAI) and those that are highly reactive and opportunistic but lack structured planning. Static systems are inflexible, unable to adapt to individual student needs or unexpected events. Reactive systems, while adaptable, can be verbose and may not efficiently cover curriculum objectives. The challenge is to create ITS that can both plan comprehensive instruction and adapt dynamically to student behavior and real-world constraints like time. This paper aims to bridge this gap by developing a planner that can generate, execute, and dynamically replan instructional strategies.
Methods
The core of the BB-IP is its blackboard architecture, a common AI approach for managing complex, multi-agent systems. In BB-IP, this architecture is used to represent and manipulate the instructional plan. Key components include:
- Instructional Plan Blackboard: Stores the hierarchical plan, broken down into objectives, activities, and specific procedures. This representation includes the rationale behind each step, crucial for replanning.
- Student Model Blackboard: Tracks the student's knowledge, inferred from pre-instruction questionnaires and ongoing performance.
- Device Blackboard: Holds information about the domain being taught (in this case, a hydraulic-electronic-mechanical assembly of a naval gun mount).
- Planner Control Blackboard: Manages meta-level decisions about planning and execution.
The planning process is hierarchical and top-down, starting with broad instructional objectives and progressively refining them into specific activities and executable procedures. A key innovation is dynamic replanning: the system continuously monitors execution. If a student asks a question, makes a request, or if the student model or time remaining changes unexpectedly, the planner enters an "Edit" phase to modify the current plan. This can involve adding, removing, or altering steps and parameters. The system also employs "plan critics" to ensure smooth discourse flow and avoid redundancy.
Datasets
The paper does not explicitly mention training or evaluation datasets in the traditional machine learning sense. Instead, it describes the implementation and testing of the BB-IP within a prototype tutor called the Lower Hoist Tutor. This tutor is designed to teach troubleshooting for the Mark-45 naval gun mount's lower hoist assembly. The "data" used for evaluation are the interactions and performance of students (or simulated students) within this specific domain.
Key Insights and Ideas
- Blackboard Architecture for Dynamic Planning: The paper convincingly demonstrates how a blackboard architecture, typically used for integrating diverse knowledge sources, can be effectively repurposed for dynamic instructional planning. It provides a flexible framework for representing, modifying, and executing complex plans.
- Integrating Plan-Based and Opportunistic Teaching: BB-IP aims to combine the benefits of structured, goal-oriented lesson planning with the ability to seize teachable moments and respond to student-initiated interactions.
- Rationale-Driven Replanning: By explicitly representing the rationale behind instructional steps (i.e., why a step is being performed), the planner can make more informed decisions when revising plans, ensuring that modifications align with overall instructional goals.
- Hierarchical Plan Representation: The three-level representation (objectives, activities, procedures) allows for both high-level strategic planning and low-level tactical execution, facilitating incremental planning and easier modification.
- Mixed-Initiative Support: The system is designed to handle student questions and requests gracefully, allowing for a more collaborative and less rigid learning experience.
Key Results
The paper presents a functional prototype, the Lower Hoist Tutor, demonstrating the BB-IP's capabilities. While quantitative results are not the primary focus, the paper highlights:
- Customization: The BB-IP can generate significantly different lesson plans for students with varying backgrounds (e.g., "Electronics-Deficient, Low-Aptitude" vs. "Has-Prerequisites, High-Aptitude"), tailoring the depth and pace of instruction.
- Dynamic Replanning in Action: The paper provides detailed examples of how the planner can splice in steps to answer student questions, review material based on inferred knowledge gaps, and adjust plans due to time constraints.
- Integration: The BB-IP successfully integrates with other ITS components (student model, domain expert) to control instruction.
- Two Hours of Instruction: The Lower Hoist Tutor, powered by BB-IP, can deliver approximately two hours of instruction on troubleshooting the lower hoist assembly.
Limitations and Future Work
- Domain Specificity: While the BB-IP architecture is designed to be generic, its implementation and testing are tied to the specific domain of the Lower Hoist Tutor. Applying it to other domains would require adapting the domain knowledge and potentially the plan critics.
- Educational Psychology Guidance: The authors acknowledge that there is "little guidance from the educational psychology literature specific enough" to make definitive choices about how to customize plans or when and how to replan in all situations. This suggests that future work could benefit from stronger theoretical grounding in learning sciences.
- Diagnosis Methods: The implemented diagnosis methods for ineffective instruction are limited. Expanding these would improve the system's ability to recover from instructional failures.
- Resource Management: While mentioned as a consideration, the paper notes that the first version of the planner focused less on resource management. Further development in this area could enhance its practical applicability.
The authors suggest that the BB-IP framework itself is robust and could support alternative pedagogical approaches, indicating a clear path for future research in refining the planning strategies and expanding its application.
Summary updated