Expressing Concern

Marc LuriaACL 1987 · DOI

Introduction

In the development of early AI "consultant systems," such as the UNIX Consultant (UC), the primary objective was to assist users in navigating complex software environments. However, a significant safety and usability gap emerged: most classical planners (e.g., STRIPS or NOAH) focused exclusively on finding a logical path to a goal without considering the risks inherent in the plan. Providing a technically correct instruction—such as "Type rm * to clear the directory"—without mentioning that the action is irreversible is fundamentally dangerous.

This paper introduces the Knowledge Intensive Planner (KIP), a system designed to bridge this gap by evaluating potential plan failures and determining which caveats are necessary to communicate to the user. By utilizing a declarative structure known as a "concern," KIP balances the need for safety with Grice’s Maxim of Quantity, ensuring users are informed of risks without being overwhelmed by trivial details.

The Problem: Binary Planning and Communication Gaps

Classical AI planning traditionally treated plan failure as a binary search problem: if a precondition was not met, the plan simply failed. This approach lacked three critical capabilities necessary for human-centric consultation:

  1. Risk Evaluation: The ability to determine the likelihood or impact of a potential failure.
  2. Communicative Nuance: The ability to decide which potential problems are worth mentioning based on the context of the interaction.
  3. Uncertainty Management: The ability to plan effectively even when the state of the world (e.g., whether a printer is currently online) is unknown.

KIP addresses these issues by moving away from purely procedural search toward a knowledge-based approach that treats potential failures as first-class objects.

The KIP Architecture and the "Concern"

The core innovation of KIP is the Concern, a declarative knowledge structure implemented in the KODIAK representation language. Unlike procedural planners that discover failures during execution or tree-traversal, KIP uses concerns to link specific plans to potential failures and long-term user interests.

Interests vs. Goals

KIP distinguishes between a user's immediate goals (e.g., "I want to print this file") and their long-term interests (e.g., "I want to preserve my data" or "I want to avoid wasting resources"). Concerns typically arise when a plan designed to achieve a goal threatens a long-term interest. Because these concerns are declarative, they can be inherited through a hierarchy of commands, allowing the system to understand that risks associated with "writing to a file" apply to any command that modifies disk contents.

The Three-Tiered Treatment Algorithm

To manage these concerns, KIP employs a three-tiered algorithm that determines how to handle a detected risk based on its "degree of concern" (ranked on a heuristic scale of 1-10):

  • Disregard: If the degree of concern is negligible (e.g., a highly improbable hardware failure), KIP ignores it. This prevents the system from cluttering the user interface with "common sense" noise.
  • Overlook & Express: If a concern is moderate, KIP suggests the plan but explicitly attaches a warning. For example, if a user asks to print, KIP might suggest the lpr command but add, "This plan will not work if the printer is out of paper."
  • Elevate: If the concern is high—representing a high probability of failure or a catastrophic side effect—KIP treats the concern as a hard failure. It then attempts to find an alternative plan or modify the current one to mitigate the risk.

Qualitative Results in the UNIX Domain

The utility of KIP is demonstrated through several practical scenarios within the UNIX environment:

  • File Permissions: If a user attempts to edit a file they do not own, KIP recognizes a "violated default concern" regarding write permissions. Because this is a high-degree concern, KIP "elevates" it, suggesting the user copy the file to their own directory first rather than simply providing the command for the editor.
  • Resource Management: When asked how to print, KIP identifies the potential for the printer to be offline or out of paper. It "overlooks" these as blockers to the plan itself but "expresses" them as caveats in the natural language output.
  • Computational Efficiency: By using a heuristic "degree of concern" to prune the search space, KIP avoids the computational explosion that occurs when a system tries to exhaustively check every possible physical precondition (e.g., checking if the computer might explode or if the power might fail).

Limitations and Future Directions

While KIP represents a significant step toward safer AI consultation, it faces a notable knowledge engineering bottleneck. The current system relies on human experts to manually curate the knowledge base of concerns and interests. Furthermore, the 1-10 ranking system for concerns is static and does not yet adapt to varying levels of user expertise.

Future work suggests that KIP could be integrated with learning components, such as the UCTeacher program, to automatically detect the frequency of plan failures and dynamically update the "degree of concern" based on real-world experience. This would allow the system to evolve from a static expert system into a dynamic, learning consultant.

Summary updated

BibTeX
@inproceedings{luria1987expressing,
  author = {Marc Luria},
  title = {{Expressing Concern}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {1987},
  doi = {10.3115/981175.981206},
}
Figure