An engineering approach to knowledge acquisition by the interactive analysis of dictionary definitions

Soujanya PoriaDurham 1998

Abstract

It has long been recognised that everyday dictionaries are a potential source of lexical and world knowledge of the type required by many Natural Language Processing (NLP) systems. This research presents a semi-automated approach to the extraction of rich semantic relationships from dictionary definitions. The definitions are taken from the recently published "Cambridge International Dictionary of English" (CIDE). The thesis illustrates how many of the innovative features of CIDE can be exploited during the knowledge acquisition process. The approach introduced in this thesis uses the LOLITA NLP system to extract and represent semantic relationships, along with a human operator to resolve the different forms of ambiguity which exist within dictionary definitions. Such a strategy combines the strengths of both participants in the acquisition process: automated procedures provide consistency in the construction of complex and inter-related semantic relationships, while the human participant can use his or her knowledge to determine the correct interpretation of a definition. This semi-automated strategy eliminates the weakness of many existing approaches because it guarantees feasibility and correctness: feasibility is ensured by exploiting LOLITA's existing NLP capabilities so that humans with minimal linguistic training can resolve the ambiguities within dictionary definitions; and correctness is ensured because incorrectly interpreted definitions can be manually eliminated. The feasibility and correctness of the solution is supported by the results of an evaluation which is presented in detail in the thesis.

Introduction

The "knowledge acquisition bottleneck" remains a significant hurdle in Artificial Intelligence. For Natural Language Processing (NLP) systems to achieve true comprehension, they require vast repositories of lexical and world knowledge. Historically, researchers have faced a binary choice: the labor-intensive manual coding of knowledge bases (as seen in projects like Cyc) or fully automated extraction from dictionaries, which often results in high error rates due to the inherent ambiguity of human language.

This research proposes a "Natural Language Engineering" (NLE) approach to bridge this gap. By combining the deep parsing capabilities of the LOLITA NLP system with human operators to resolve linguistic ambiguities, the research demonstrates a feasible way to extract rich, accurate semantic relationships from the Cambridge International Dictionary of English (CIDE).

Methodology

The author adopts an NLE philosophy, prioritizing pragmatic, scalable, and maintainable solutions over purely theoretical ones.

The LOLITA System and SemNet

The research utilizes LOLITA (Large-Scale Object-based Linguistic Interactor, Translator and Analyser), an NLP system built around a semantic network called SemNet. WordNet (version 1.4) was used as the initial "seed" for LOLITA’s SemNet.

Source Material: CIDE

The Cambridge International Dictionary of English was selected due to two innovative features:

  1. Defining Vocabulary: It uses a restricted set of only 2,000 words for all definitions, simplifying the disambiguation process.
  2. Entry Layout: It groups semantically related words (e.g., the verb manufacture and the noun manufacturer) in the same entry, providing implicit context.

Interactive Disambiguation

To make dictionary fragments readable for LOLITA, the system transforms them into full sentences (e.g., "Each [noun] is [definition]"). A Windows-based GUI presents points of ambiguity to non-expert operators. Instead of requiring linguistic expertise, the system uses a "rejection" strategy: it shows different bracketed versions of a sentence and asks the user to reject those that do not make sense.

Key Insights

  • The Power of Rejection: Humans find it much easier to identify a wrong interpretation than to construct a correct one from scratch. By presenting multiple parses and asking the user to "reject the nonsense," the system achieves high accuracy without requiring the user to be a linguist.
  • Implicit Entities: Dictionary definitions often omit the "actor" (e.g., an abattoir is a place where animals are killed, but the definition does not specify who kills them). The system identifies these gaps and prompts the operator to name the implicit entity (e.g., a "butcher").
  • NLE vs. NLP: The paper emphasizes "Engineering" over "Processing," focusing on preventing the "snowball effect" of errors. Human verification ensures that incorrect knowledge does not propagate through the semantic hierarchy.

Results

The system was evaluated using ten operators (primarily software engineers) who analyzed a set of 50 words (30 nouns and 20 verbs) chosen for their linguistic complexity.

  • Word Sense Disambiguation: 97.9% accuracy.
  • Pronoun Resolution: 100% success rate.
  • Parse Tree Selection: 71.5% accuracy (projected to reach 86.2% with minor UI refinements).
  • Overall Feasibility: On average, 87% of all questions were answered correctly by untrained operators, proving that the system can be used by non-experts to build a high-quality knowledge base.

Conclusion and Future Work

While more efficient than manual coding, processing 50,000+ definitions still requires significant human effort. Future developments should focus on contextual processing, where definitions are processed in the context of previously entered ones to automatically constrain new interpretations. Additionally, certain common dictionary constructs like "especially X" or "such as X" require better formalization within the semantic network to improve extraction accuracy.

Summary updated

Abstract

It has long been recognised that everyday dictionaries are a potential source of lexical and world knowledge of the type required by many Natural Language Processing (NLP) systems. This research presents a semi-automated approach to the extraction of rich semantic relationships from dictionary definitions. The definitions are taken from the recently published "Cambridge International Dictionary of English" (CIDE). The thesis illustrates how many of the innovative features of CIDE can be exploited during the knowledge acquisition process. The approach introduced in this thesis uses the LOLITA NLP system to extract and represent semantic relationships, along with a human operator to resolve the different forms of ambiguity which exist within dictionary definitions. Such a strategy combines the strengths of both participants in the acquisition process: automated procedures provide consistency in the construction of complex and inter-related semantic relationships, while the human participant can use his or her knowledge to determine the correct interpretation of a definition. This semi-automated strategy eliminates the weakness of many existing approaches because it guarantees feasibility and correctness: feasibility is ensured by exploiting LOLITA's existing NLP capabilities so that humans with minimal linguistic training can resolve the ambiguities within dictionary definitions; and correctness is ensured because incorrectly interpreted definitions can be manually eliminated. The feasibility and correctness of the solution is supported by the results of an evaluation which is presented in detail in the thesis.
BibTeX
@inproceedings{poria1998engineering,
  author = {Soujanya Poria},
  title = {{An engineering approach to knowledge acquisition by the interactive analysis of dictionary definitions}},
  booktitle = {Durham},
  year = {1998},
}
Figure