Case-Based Diagnostic Analysis in a Blackboard Architecture

Edwina L. Rissland, Jody J. Daniels, Zachary B. Rubinstein, David B. SkalakAAAI 1993

Introduction

This paper introduces FRANK (Flexible Report and Analysis System), a hybrid AI system that integrates Case-Based Reasoning (CBR) into a blackboard architecture to generate tailored diagnostic reports. Unlike traditional systems that treat case retrieval as a static process, FRANK dynamically adjusts its reasoning strategies and high-level expository goals based on the cases available in its memory.

The Challenge of Intent-Driven Retrieval

In traditional Case-Based Reasoning, systems typically search for the "most similar" cases to solve a problem. However, this "one-size-fits-all" approach often ignores the user’s intent. For instance, a doctor writing a neutral medical memo requires a different set of cases and analysis than a lawyer drafting an advocacy brief.

Furthermore, CBR systems often struggle when they fail to find a "perfect" match. FRANK addresses this by intelligently responding to retrieval failure—not just by loosening search parameters, but by "negotiating" with the user to modify the report's goal if the available data does not support the original intent.

System Architecture

FRANK is built using a blackboard architecture (specifically the Generic Blackboard toolkit, GBB), allowing diverse "knowledge sources"—including CBR modules, OPS5 rule-based systems, and planners—to collaborate on a shared data space.

High-level architecture of the FRANK system showing the integration of planning and domain reasoning modules.

The system's operation is governed by three distinct hierarchies:

  1. Report Hierarchy: Categorizes reports by their "stance" (e.g., Argumentative vs. Summarizing).
  2. Problem-Solving Strategies Hierarchy: Encodes analytical methods (e.g., "emphasize strengths").
  3. Presentation Strategies Hierarchy: Dictates the final rhetorical structure.

Hierarchical Reasoning and Intent

The Report Hierarchy is central to FRANK's ability to tailor its output. By defining the stance of the report early on, the system can select appropriate cases that serve that specific rhetorical goal.

The Report Hierarchy used by FRANK to categorize diagnostic stances.

FRANK utilizes a "vectored interrupt" feedback mechanism. If a low-level task, such as finding a specific case, fails, an interrupt is triggered. The system first attempts local recovery (e.g., broadening the search). If that fails, the interrupt bubbles up to higher levels, potentially prompting the system to select a new plan or ask the user to change the report type entirely.

Key Insights

  • Goal-Dependent Retrieval: The definition of a "best case" is fluid, depending on whether the goal is a balanced overview or a persuasive argument.
  • Failure as Information: Retrieval failure is treated as a signal. If cases cannot be found to support a specific position, it indicates that the position itself may be weak, prompting a shift in strategy.
  • Dynamic Re-specification: FRANK can modify its "memory probe" on the fly, changing similarity metrics or definitions of "on-point" cases based on intermediate results.
  • Hybrid Reasoning: By combining CBR with rule-based reasoning (RBR), the system leverages rules for quick inferences and CBR for the precedent-based justification required in professional reports.

Application and Results

The primary application domain for FRANK is back-injury diagnosis, specifically spinal conditions like spinal stenosis. In a qualitative evaluation, the authors demonstrated FRANK's ability to handle unsustainable user requests. For example, when a user requested a "pro-position" report for a diagnosis that lacked clear evidentiary support in the case base, FRANK successfully triggered a global interrupt. It then suggested switching to a "Diagnosis-Own-Merit" report, which focused on the case's individual strengths rather than an unsupported comparison with alternatives.

Limitations and Future Work

While FRANK represents a significant step toward flexible diagnostic systems, several areas for improvement remain:

  • Automation: Future versions could automate the initial report type selection based on the problem description.
  • Domain Engineering: Porting the system to other fields like law or policy requires significant manual engineering of hierarchies and strategies.
  • Natural Language Generation: The current system uses templates; integrating more sophisticated linguistic realization would improve the flow of generated arguments.

Summary updated

BibTeX
@inproceedings{rissland1993case,
  author = {Edwina L. Rissland and Jody J. Daniels and Zachary B. Rubinstein and David B. Skalak},
  title = {{Case-Based Diagnostic Analysis in a Blackboard Architecture}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1993},
}
Figure