A Tool for Achieving Consensus in Knowledge Representation

Loren G. Terveen, David A. WroblewskiAAAI 1991

Abstract

Building large-scale knowledge bases (KBs) is a collaborative design challenge rather than a simple data entry task. This paper introduces the HITS Knowledge Editor (HKE), a tool designed to facilitate consensus among editors across both time and space. By employing a visual sketching interface and automated "design critics," HKE helps maintain consistency and captures design intent within complex systems like the CYC knowledge base.

The Challenge of Consensus in Knowledge Engineering

In the early 1990s, projects like CYC aimed to encode vast amounts of human knowledge into machine-readable formats. A recurring problem identified by the authors was the lack of a "single correct representation" for many domains. For instance, designers might disagree on whether "Sex" should be a slot on a generic "Person" object or if "Male" and "Female" should be distinct subclasses.

When multiple editors contribute to a shared KB, conflicting design choices are inevitable. Without consensus—defined here as a mutual understanding of the rationale behind a decision—the KB becomes a tangled web of inconsistencies. The HKE addresses two types of consensus:

  1. Synchronous Consensus: Helping a team in the same room agree on a design.
  2. Asynchronous Consensus: Helping new editors understand and respect the design decisions made by predecessors who may have left the project years ago.

The HITS Knowledge Editor (HKE)

The HKE acts as an intelligent interface to the CYC knowledge base, supporting consensus through three primary functional roles.

1. The Design Medium (Sketching)

Instead of direct KB manipulation, users work in a visual "sketchpad." They draw graphs representing objects and relationships, creating a buffer where ideas can be visualized and debated before being formally committed. This visual representation serves as a shared mental model; in user studies, seeing an inconsistent graph prompted subjects to resolve the issue immediately.

The HKE sketching interface being used by subjects to map out a research laboratory structure.

2. The Intelligent Assistant (Critics)

When a user attempts to incorporate a sketch into the KB, HKE runs automated design critics. These agents check the new input against established KB conventions. If a violation occurs—such as attempting to make an object an instance of two mutually disjoint classes—the system flags a "trouble" and provides repair suggestions. This mechanism is crucial for maintaining asynchronous consensus with the original KB architects.

An HKE "Trouble" dialog explaining a logical conflict and offering a "repair resource" to guide the user back into alignment with established design.

3. The Design Recorder

HKE saves sketches as a side effect of the design process. These saved visual artifacts allow future users to "recover" the design intent and vocabulary used by previous designers, providing low-overhead documentation without forcing engineers to perform the "off-task" chore of manual writing.

Evaluation and Results

Qualitative user studies involving the representation of the MCC AI Lab structure demonstrated the tool's effectiveness:

  • Synchronous Case: Subjects used the sketching tool to identify and resolve inconsistent mapping of researchers to organizations over a 10-minute collaborative session.
  • Asynchronous Case: When subjects attempted to represent programs as instances (violating CYC's subclass convention), the HKE critics successfully detected the "trouble" and guided them back into alignment with the established hierarchy.

Limitations and Future Work

While HKE successfully captures what was decided, it does not yet explicitly record the why (the underlying arguments). Additionally, the tool currently lacks support for synchronous distributed collaboration, where geographically separated users work on the same KB simultaneously. Future work may involve "agents" to notify users of concurrent edits in related KB sections.

Summary updated

BibTeX
@inproceedings{terveen1991tool,
  author = {Loren G. Terveen and David A. Wroblewski},
  title = {{A Tool for Achieving Consensus in Knowledge Representation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1991},
}
Figure