Tailoring Retrieval to Support Case-Based Teaching

Robin D. Burke, Alex KassAAAI 1994

Abstract

Traditional information retrieval systems are passive, requiring users to formulate specific queries. In educational contexts, particularly for complex social tasks like sales or diplomacy, students often lack the domain knowledge to know what they need to ask. This paper introduces SPIEL (Story Producer for InteractivE Learning), an "active" retrieval system that monitors student performance in social simulations and proactively provides relevant expert video stories at optimal pedagogical moments.

The Challenge of Passive Retrieval in Education

Classical Information Retrieval (IR) models, such as library searches, assume a user can articulate their information needs. However, novices learning nuanced social skills frequently encounter "unknown unknowns." A student might successfully close a deal while inadvertently taking a significant risk, missing a crucial lesson because they didn't realize a cautionary tale was relevant.

To address this, an educational retrieval system must:

  1. Initiate retrieval autonomously without a user query.
  2. Determine educational relevance, recognizing that a story's value lies in its pedagogical utility—such as providing a counter-example to a recent success—rather than just keyword similarity.

The SPIEL Methodology

SPIEL is integrated within GuSS (Guided Social Simulation), a framework for practicing social interactions. The system operates through two primary phases: storage and retrieval.

Storage and Indexing

Human experts index video stories using a structured language centered on anomalies or expectation violations. These indices follow a specific template: Actor X expected Y, but Z happened. This structure allows the system to map stories to specific student misconceptions or performance gaps.

System Architecture

The core of SPIEL involves 13 storytelling strategies (e.g., "Demonstrate risks," "Critique expectations") that act as templates for intervention. The system automatically generates Recognition Condition Descriptions (RCDs) that bridge high-level story indices with low-level simulation events.

Figure 2: The SPIEL architecture showing the flow from story storage and index generation to real-time retrieval based on simulation events.

The retrieval phase functions as a production system, matching RCDs against the real-time state of the simulation to identify "teachable moments." These RCDs follow a "Trigger-Evidence-Presentation" logic, allowing the system to "pounce" on teaching opportunities as they arise during the simulation.

User Experience: The "Lurking" Tutor

By moving retrieval to the background, SPIEL mimics a real-world mentor who watches over a trainee's shoulder. When a student triggers a specific pedagogical condition, the system interrupts to present a "war story" from an experienced professional.

Figure 1: The Storyteller interface, displaying a video story alongside the pedagogical rationale ("Why the Storyteller found this story") and suggested takeaways ("What you can do").

The interface provides context for the intervention, explaining why the story was chosen and what the student should learn from it. This "bridge and coda" approach ensures the student understands the relevance of the expert's experience to their own current simulation state, turning a static video clip into a dynamic lesson.

Key Insights

  • Pedagogical Relevance vs. Semantic Similarity: A story is relevant if it shifts a student's perspective. SPIEL might retrieve a failure story precisely because a student succeeded, acting as a prophylactic against overconfidence.
  • Anomalies as Hooks: Expert stories are inherently interesting due to surprises. Indexing based on expectation violations allows the system to target student misconceptions effectively.

Conclusion and Future Work

Formative evaluations with professional account executives indicated that SPIEL's interventions were perceived as highly relevant and engaging. While technically feasible, the system faces a "knowledge bottleneck," requiring significant manual effort to build domain-specific knowledge bases. Future research aims to develop tools that allow subject matter experts to easily apply these storytelling strategies to their own video libraries without deep AI expertise.

Summary updated

BibTeX
@inproceedings{burke1994tailoring,
  author = {Robin D. Burke and Alex Kass},
  title = {{Tailoring Retrieval to Support Case-Based Teaching}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1994},
}

Loading related content...

Figure