Introduction
This paper introduces FRANK (Flexible Report and Analysis System), a hybrid AI system that integrates Case-Based Reasoning (CBR) into a blackboard architecture to generate tailored diagnostic reports. Unlike traditional systems that treat case retrieval as a static process, FRANK dynamically adjusts its reasoning strategies and high-level expository goals based on the cases available in its memory.
The Challenge of Intent-Driven Retrieval
In traditional Case-Based Reasoning, systems typically search for the "most similar" cases to solve a problem. However, this "one-size-fits-all" approach often ignores the user’s intent. For instance, a doctor writing a neutral medical memo requires a different set of cases and analysis than a lawyer drafting an advocacy brief.
Furthermore, CBR systems often struggle when they fail to find a "perfect" match. FRANK addresses this by intelligently responding to retrieval failure—not just by loosening search parameters, but by "negotiating" with the user to modify the report's goal if the available data does not support the original intent.
System Architecture
FRANK is built using a blackboard architecture (specifically the Generic Blackboard toolkit, GBB), allowing diverse "knowledge sources"—including CBR modules, OPS5 rule-based systems, and planners—to collaborate on a shared data space.
The system's operation is governed by three distinct hierarchies:
- Report Hierarchy: Categorizes reports by their "stance" (e.g., Argumentative vs. Summarizing).
- Problem-Solving Strategies Hierarchy: Encodes analytical methods (e.g., "emphasize strengths").
- Presentation Strategies Hierarchy: Dictates the final rhetorical structure.
Hierarchical Reasoning and Intent
The Report Hierarchy is central to FRANK's ability to tailor its output. By defining the stance of the report early on, the system can select appropriate cases that serve that specific rhetorical goal.
FRANK utilizes a "vectored interrupt" feedback mechanism. If a low-level task, such as finding a specific case, fails, an interrupt is triggered. The system first attempts local recovery (e.g., broadening the search). If that fails, the interrupt bubbles up to higher levels, potentially prompting the system to select a new plan or ask the user to change the report type entirely.
Key Insights
- Goal-Dependent Retrieval: The definition of a "best case" is fluid, depending on whether the goal is a balanced overview or a persuasive argument.
- Failure as Information: Retrieval failure is treated as a signal. If cases cannot be found to support a specific position, it indicates that the position itself may be weak, prompting a shift in strategy.
- Dynamic Re-specification: FRANK can modify its "memory probe" on the fly, changing similarity metrics or definitions of "on-point" cases based on intermediate results.
- Hybrid Reasoning: By combining CBR with rule-based reasoning (RBR), the system leverages rules for quick inferences and CBR for the precedent-based justification required in professional reports.
Application and Results
The primary application domain for FRANK is back-injury diagnosis, specifically spinal conditions like spinal stenosis. In a qualitative evaluation, the authors demonstrated FRANK's ability to handle unsustainable user requests. For example, when a user requested a "pro-position" report for a diagnosis that lacked clear evidentiary support in the case base, FRANK successfully triggered a global interrupt. It then suggested switching to a "Diagnosis-Own-Merit" report, which focused on the case's individual strengths rather than an unsupported comparison with alternatives.
Limitations and Future Work
While FRANK represents a significant step toward flexible diagnostic systems, several areas for improvement remain:
- Automation: Future versions could automate the initial report type selection based on the problem description.
- Domain Engineering: Porting the system to other fields like law or policy requires significant manual engineering of hierarchies and strategies.
- Natural Language Generation: The current system uses templates; integrating more sophisticated linguistic realization would improve the flow of generated arguments.
Summary updated