Case-based Reasoning

IJCAI 2001

TL;DR

This paper introduces "monitored distribution," a novel approach to knowledge management that tightly integrates lessons learned (LL) into decision-making processes. By embedding LL systems within existing tools, it aims to bridge the "lesson distribution gap" and significantly improve plan evaluation measures in domains like military planning.

What Problem Does This Paper Address?

Organizations collect valuable "lessons learned" (LL) from past experiences to improve future decision-making. However, existing LL systems often fail to deliver these lessons to users when and where they are most needed, leading to underutilization. This is termed the "lesson distribution gap." Traditional methods rely on users actively searching for lessons (pulling) or receive them through less integrated means (pushing), which are often ineffective due to unstructured formats, the need for users to master separate systems, and the challenge of applying lessons correctly. The paper addresses how to proactively and effectively deliver LL to users within their existing workflows.

Methods

The core contribution is the monitored distribution approach. This involves:

  • Tight Integration: Embedding LL systems directly within decision support tools.
  • Monitoring: Inserting a "monitor" into the decision process to detect situations where a lesson might be applicable.
  • Proactive Delivery: Presenting relevant lessons to the user at the opportune moment.
  • Case-Based Reasoning (CBR) Implementation: The paper details a CBR implementation called ALDS (Active Lesson Delivery System), which is a module of the HICAP plan authoring tool suite. ALDS uses cases to represent lessons, with indexing components (target task, applicability conditions) and reuse components (suggestion, explanation). This allows for partial matching of lessons to current situations.

Datasets

  • Simulated Military Planning Domain: The evaluation was conducted in a simulated Noncombatant Evacuation Operations (NEO) domain.
  • Active Navy Lesson Repository: 11 NEO-related lessons were selected from a repository of 5120 lessons for the experiment.

Key Insights and Ideas

  • The "Lesson Distribution Gap" is a Real Problem: The paper highlights that simply collecting lessons isn't enough; effective distribution and reuse are critical for knowledge management to be impactful.
  • Integration is Key: Proactively bringing lessons to users within their existing workflow (monitored distribution) is more effective than relying on users to search for them.
  • CBR is Well-Suited for LL: The partial matching capabilities of CBR make it ideal for indexing and retrieving lessons based on their applicability conditions.
  • Monitored Distribution Improves Performance: The core hypothesis is that this integrated approach leads to better outcomes in decision-making tasks.

Key Results

The evaluation in the simulated NEO domain showed that monitored distribution (using ALDS) significantly improved plan evaluation measures. Specifically:

  • Overall Duration: Reduced by approximately 18%.
  • Duration until Medical Assistance: Reduced by approximately 18%.
  • Casualties to Evacuees: Reduced by 24% on average.
  • Casualties to Friendly Forces: Reduced by 30%.
  • An average of three lessons were used per plan.

Limitations and Future Work

  • Simulated Users: The evaluation used simulated users, not actual operational planners, which limits the direct applicability of findings to real-world scenarios.
  • Availability of Lessons: The experimental design ensured lessons were available for prompting, which may not always be the case in real-world systems.
  • Future Work:
    • Characterize conditions under which monitored distribution yields the most significant improvements.
  • Conduct subject experiments comparing monitored distribution with traditional keyword search tools.
  • Demonstrate that monitored distribution is not restricted to planning tasks.
  • Develop lesson collection tools to reduce knowledge engineering effort.
  • Explore interactive elicitation approaches for populating LL repositories.

Summary updated

BibTeX
@inproceedings{2001case,
  title = {{Case-based Reasoning}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2001},
}
Figure