Introduction
Traditional Intelligent Tutoring Systems (ITS) often rely on "shallow" student models that track mastery of specific topics but fail to capture the underlying mental processes or "deep" logic a student uses to reason about a system. This paper introduces PRESTO, an ITS designed to model a student's qualitative mental model of physical devices. By moving beyond simple correctness tracking, PRESTO aims to diagnose deep-seated misconceptions and provide pedagogically sound feedback through analogies and simulations.
Qualitative Reasoning and Confluences
The core of PRESTO's modeling capability lies in Qualitative Reasoning (QR), specifically utilizing deKleer and Brown’s theory of confluences. Humans typically reason about physical systems—such as pressure regulators or leaky pipes—qualitatively rather than through complex numerical fluid dynamics.
Relationships between physical variables are represented as confluence equations. These equations describe how changes (represented by the partial derivative symbol $\partial$) in one variable relate to changes in others without requiring precise numerical values. For example, a simple relationship might be expressed as $\partial \text{pressure} - \partial \text{flow} = 0$.
The PRESTO System Architecture
The system operates through a multi-stage process to build and test a "runnable" student model:
- Model Extraction: PRESTO presents a diagram of a device and prompts the student to describe causal relationships between its components.
- Constraint Satisfaction: The student's stated beliefs are converted into a model. A constraint satisfaction algorithm propagates changes through this logic to predict the student's answers to specific scenarios.
- Diagnosis: By comparing the output of the student's faulty model against a "correct" expert model, PRESTO identifies the specific "malformed" confluence equation representing the misconception.
Remediation Strategies
Once a misconception is identified, PRESTO employs several strategies to correct the student's mental model. A primary method is the use of analogies. The system presents a simpler, more intuitive device—such as a pipe with a constriction—to highlight the specific physical relationship the student misunderstood in the more complex device.
Other strategies include contradiction, where the system demonstrates how the student's logic leads to physically impossible outcomes, and simulation, which allows students to visualize the device behaving according to their own incorrect beliefs.
Key Insights and Results
An experimental study with 12 students demonstrated that PRESTO could successfully identify "deep" misconceptions, particularly those involving complex interactions like feedback loops. The study validated that "malformed" confluences are an effective way to represent human errors in a computer-readable format.
However, a significant technical challenge identified was the symmetry vs. causality gap. While confluence equations are mathematically symmetric, human reasoning is inherently causal and directed. For instance, a student might correctly understand that closing a valve increases pressure, but incorrectly assume that increasing pressure must close the valve. Future work aims to incorporate directed causal information to address this limitation and streamline the model extraction phase.
Conclusion
PRESTO demonstrates the power of qualitative reasoning in student modeling. By creating "runnable" models of student beliefs, ITS can provide more nuanced and effective instruction, moving closer to the goal of truly understanding the learner's internal logic.
Summary updated