Recovery from Incorrect knowledge in Soar

John E. LairdAAAI 1988

Abstract

In the Soar cognitive architecture, long-term knowledge is traditionally "add-only," meaning once a rule is learned via chunking, it cannot be deleted or modified. This paper explores how Soar can recover from incorrect or overgeneralized knowledge by learning new rules that override erroneous decisions. By leveraging existing architectural mechanisms like impasses and preference logic, the system can effectively "out-reason" its past mistakes without requiring a deletion mechanism.

Introduction

A significant challenge in cognitive architectures like Soar is the persistence of incorrect knowledge. In Soar, all long-term knowledge is stored as production rules, and learning occurs through "chunking." Because the architecture does not support deleting or weakening these rules, an overgeneralized or incorrect rule can lead to persistent errors. This paper addresses whether an "add-only" memory system can recover from its own mistakes by focusing on decision-making rather than knowledge modification.

Methodology: The Recovery Cycle

Instead of attempting to fix the underlying rules, the proposed approach fixes the decisions the system makes. This is achieved through a five-phase cycle:

  1. Detection: The system identifies an error, such as reaching an illegal state or receiving negative feedback.
  2. Forcing Reconsideration: To trigger an impasse, the system deliberately creates a conflict in its decision-making process. This forces Soar into a subgoal to resolve the conflict.
  3. Reconsidering the Decision: Within the subgoal, Soar performs a look-ahead search in a "selection problem space," evaluating alternatives more rigorously.
  4. Determining the Correction: If the original choice is found to be suboptimal, a "reject" preference is created. If a previously blocked good choice is found, an "indirect" operator is used to bypass the block.
  5. Saving the Correction: Soar’s chunking mechanism automatically converts this reconsideration into a new production rule.
The five-phase recovery cycle in Soar, illustrating the transition from a conflict-impasse to a legal state through look-ahead evaluation.

In subsequent encounters, the new correction rule fires in parallel with the original incorrect rule. Soar's preference logic ensures the correction takes precedence, effectively overriding the error.

Evaluation and Results

The recovery mechanism was tested on three classic AI tasks: the Missionaries and Cannibals puzzle, multi-column subtraction, and the Eight Puzzle.

Performance comparison (in number of decisions) across different tasks before, during, and after the recovery process.

The results demonstrate that while there is an initial performance "tax" during the recovery phase (where more decisions are required to figure out the fix), the system eventually optimizes its behavior. In the Eight Puzzle, where incorrect knowledge initially made the task unsolvable, the recovery mechanism allowed the system to find a solution and eventually "chunk" it down to a highly efficient 12 decisions—better than the performance without incorrect knowledge.

Discussion and Limitations

This approach highlights that "unlearning" is not strictly necessary for behavioral improvement. By shifting the focus to decision-level corrections, Soar maintains its architectural simplicity while gaining flexibility. The system utilizes its inherent mechanisms—impasses, subgoals, and parallel rule firing—to facilitate self-correction.

However, limitations remain. The system relies on the ability to detect errors and requires a reliable "verifier" (like look-ahead search) to determine the correct path. If the detection mechanism or the verifier is flawed, the system may still fail to recover. Future work aims to expand the range of detectable errors and improve the scalability of these recovery techniques beyond toy problems.

Summary updated

BibTeX
@inproceedings{laird1988recovery,
  author = {John E. Laird},
  title = {{Recovery from Incorrect knowledge in Soar}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1988},
}
Figure