Student Modelling with Confluences

Daniel Baril, Jim E. Greer, Gordon I. McCallaAAAI 1991

Introduction

Traditional Intelligent Tutoring Systems (ITS) often rely on "shallow" student models that track mastery of specific topics but fail to capture the underlying mental processes or "deep" logic a student uses to reason about a system. This paper introduces PRESTO, an ITS designed to model a student's qualitative mental model of physical devices. By moving beyond simple correctness tracking, PRESTO aims to diagnose deep-seated misconceptions and provide pedagogically sound feedback through analogies and simulations.

Qualitative Reasoning and Confluences

The core of PRESTO's modeling capability lies in Qualitative Reasoning (QR), specifically utilizing deKleer and Brown’s theory of confluences. Humans typically reason about physical systems—such as pressure regulators or leaky pipes—qualitatively rather than through complex numerical fluid dynamics.

Relationships between physical variables are represented as confluence equations. These equations describe how changes (represented by the partial derivative symbol $\partial$) in one variable relate to changes in others without requiring precise numerical values. For example, a simple relationship might be expressed as $\partial \text{pressure} - \partial \text{flow} = 0$.

Example of confluence equations used to represent physical relationships within the PRESTO system.

The PRESTO System Architecture

The system operates through a multi-stage process to build and test a "runnable" student model:

  1. Model Extraction: PRESTO presents a diagram of a device and prompts the student to describe causal relationships between its components.
  2. Constraint Satisfaction: The student's stated beliefs are converted into a model. A constraint satisfaction algorithm propagates changes through this logic to predict the student's answers to specific scenarios.
  3. Diagnosis: By comparing the output of the student's faulty model against a "correct" expert model, PRESTO identifies the specific "malformed" confluence equation representing the misconception.

Remediation Strategies

Once a misconception is identified, PRESTO employs several strategies to correct the student's mental model. A primary method is the use of analogies. The system presents a simpler, more intuitive device—such as a pipe with a constriction—to highlight the specific physical relationship the student misunderstood in the more complex device.

The PRESTO interface showing an analogy between a pressure regulator and a constricted pipe to remediate a student's misconception.

Other strategies include contradiction, where the system demonstrates how the student's logic leads to physically impossible outcomes, and simulation, which allows students to visualize the device behaving according to their own incorrect beliefs.

Key Insights and Results

An experimental study with 12 students demonstrated that PRESTO could successfully identify "deep" misconceptions, particularly those involving complex interactions like feedback loops. The study validated that "malformed" confluences are an effective way to represent human errors in a computer-readable format.

However, a significant technical challenge identified was the symmetry vs. causality gap. While confluence equations are mathematically symmetric, human reasoning is inherently causal and directed. For instance, a student might correctly understand that closing a valve increases pressure, but incorrectly assume that increasing pressure must close the valve. Future work aims to incorporate directed causal information to address this limitation and streamline the model extraction phase.

Conclusion

PRESTO demonstrates the power of qualitative reasoning in student modeling. By creating "runnable" models of student beliefs, ITS can provide more nuanced and effective instruction, moving closer to the goal of truly understanding the learner's internal logic.

Summary updated

BibTeX
@inproceedings{baril1991student,
  author = {Daniel Baril and Jim E. Greer and Gordon I. McCalla},
  title = {{Student Modelling with Confluences}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1991},
}
Figure