Identifying Relevant Prior Explanations

James A. RosenblumACL 1993 · DOI

Abstract

Intelligent Tutoring Systems (ITS) often struggle to maintain discourse context, resulting in repetitive and unnatural explanations that treat every student error as an isolated event. This paper introduces a Case-Based Reasoning (CBR) framework designed to identify relevant prior explanations that can serve as "anchors" for new dialogue. By utilizing a Similarity Directed Acyclic Graph (DAG) to index student actions based on expert-defined "facets," the system efficiently retrieves contextually similar past interactions. Evaluation against human-human tutorial protocols shows that this approach closely mirrors human pedagogical strategies and can even identify relevant connections that human tutors occasionally overlook.

Introduction: The Need for Discourse Context

When a human tutor guides a student, they leverage the shared history of the conversation. If a student repeats a mistake or encounters a similar logical hurdle, the tutor might say, "For the same reasons we discussed earlier..." This creates a coherent narrative and avoids redundant information.

In contrast, many automated tutors generate explanations from scratch for every query. This lack of "discourse context" makes interactions feel mechanical and can overwhelm students with repetitive content. The challenge lies in efficiently searching an ever-growing dialogue history to find relevant precedents without slowing down the real-time interaction.

Example of a human tutor using contextual anchors to refer back to previous explanations.

Methodology: The Similarity DAG

The proposed solution integrates a CBR algorithm with a text planner, specifically within the SHERLOCK tutoring environment for avionics troubleshooting. The system evaluates student actions using "facets"—multidimensional criteria used by experts to judge performance (e.g., "testing a secondary signal before a main signal").

To manage the complexity of the dialogue history, the algorithm constructs a Similarity Directed Acyclic Graph (DAG):

  • Nodes: Represent sets of previous actions sharing specific facets.
  • Structure: The graph provides a partial ordering of actions based on their similarity to the current state.
  • Efficiency: Instead of an exhaustive search, the DAG allows the system to quickly narrow down the most relevant "anchor"—the specific part of the past dialogue that explained a similar situation.
The Similarity DAG maps current student actions and facets to previous discourse history to find relevant "anchors."

Evaluation and Key Insights

The system was tested against 8 human-human tutorial transcripts containing 154 student actions and 22 instances where human tutors integrated prior explanations.

High Alignment and Precision

The algorithm demonstrated high alignment with human behavior, correctly identifying the same prior situations used by human mentors in the majority of cases. Furthermore, it showed high precision; when human tutors chose not to refer back to history, the algorithm generally found no significant similarities either.

Beyond Human Performance

Interestingly, in three cases where the algorithm suggested a similarity that the human tutor ignored, expert raters concluded that the algorithm's suggestion would have been pedagogically beneficial. This suggests that a structured CBR approach can be more consistent than human tutors in maintaining long-term discourse context.

Conclusion and Future Work

By moving beyond simple keyword matching to a facet-based similarity model, this research provides a path toward more natural AI-student interactions. Future work will focus on expanding the system's scope beyond "justification" queries and incorporating the "intentions" behind explanations to provide even more nuanced pedagogical support.

Summary updated

BibTeX
@inproceedings{rosenblum1993identifying,
  author = {James A. Rosenblum},
  title = {{Identifying Relevant Prior Explanations}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {1993},
  doi = {10.3115/981574.981613},
}

Loading related content...

Figure