Improving Rule-Based Systems Through Case-Based Reasoning

Andrew R. Golding, Paul S. RosenbloomAAAI 1991

Introduction

Building expert systems for complex, "messy" real-world domains such as law or linguistics is a significant challenge in artificial intelligence. Purely rule-based systems (RBR) often suffer from a "knowledge acquisition bottleneck," where adding rules to cover every exception leads to brittle, unmaintainable code. Conversely, purely case-based reasoning (CBR) requires an exhaustive library of examples to capture even the most basic general trends. This paper proposes a hybrid architecture that leverages the broad coverage of rules with the precision of cases to handle exceptions effectively.

Methodology: The Rules-First, Cases-as-Overrides Architecture

The authors introduce a system that uses rules for general trends and cases to "veto" those rules when a compelling exception is identified. The process follows three primary stages:

  1. Rule Proposal: Standard rules suggest a default action (an "operator").
  2. Analogy Search: The system searches a case library for "negative exemplars"—previous instances where the current rules failed.
  3. Compellingness Test: Through Inductive Verification, the system creates an "arule" (an implicit rule representing the analogy) and tests its statistical accuracy across the library. If the analogy is sufficiently "compelling," it overrides the default rule.

A critical innovation is Prediction-Based Indexing (PBI). Instead of indexing cases by surface features alone, cases are indexed by the specific rules they confirm or violate, ensuring the CBR component is only activated when rules are likely to fail.

Evaluation and Results

The system, named Anapron, was evaluated using a library of 5,000 surnames and tested against a corpus of 1.5 million US surnames. Performance was compared to commercial systems, neural networks, and human subjects.

Accuracy and Synergy

Ablation studies demonstrated that the hybrid approach significantly outperforms single-method configurations. Adding rules improved accuracy by approximately 32–38%, while the addition of cases provided a further 12–17% boost.

Accuracy gains as a function of rule and case strength.

Computational Efficiency

Contrary to the typical "utility problem" in AI—where more knowledge slows down a system—this hybrid model exhibits an efficiency synergy. Improving rule accuracy actually speeds up the CBR component by reducing the number of negative exemplars the system must evaluate.

Average processing time (seconds) per name across varying rule and case strengths.

As shown in the data, with a full case library (5,000), increasing rule strength from 0 to 1 reduced average processing time from 10.2 seconds to 7.2 seconds.

Discussion and Future Work

The hybrid approach allows for "broadly right" rules and "specifically right" cases. Key insights include the use of inductive verification as a safety net to prevent overgeneralization from single exceptions. However, the system currently relies on hand-coded similarity metrics. Future research aims to automate metric generation and allow the system to "learn" by converting successful analogies into permanent rules over time.

Summary updated

BibTeX
@inproceedings{golding1991improving,
  author = {Andrew R. Golding and Paul S. Rosenbloom},
  title = {{Improving Rule-Based Systems Through Case-Based Reasoning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1991},
}
Figure