Parsing vs. Text Processing in the Analysis of Dictionary Definitions

Thomas Ahlswede, Martha W. EvensACL 1988 · DOI

This 1988 research from the Illinois Institute of Technology explores the practical trade-offs between "deep" linguistic parsing and "shallow" text processing for converting dictionary definitions into machine-readable knowledge. While full parsing is theoretically superior for capturing complex nuances, the study concludes that simple pattern-matching is often more efficient and accurate for building the large-scale relational databases required for early AI and NLP systems.

The Problem: Building a Massive Lexicon

To develop sophisticated Natural Language Processing (NLP) systems, researchers require a massive lexicon—a database detailing how words relate to one another. For instance, a system needs to know that a "dodecahedron" is a type of "solid." Manually coding these millions of relationships is unfeasible, leading researchers to utilize machine-readable dictionaries (MRDs) like Webster’s Seventh New Collegiate Dictionary (W7).

However, dictionary definitions are written for human consumption, not computers. They contain stylistic quirks, implied information, and complex syntax. The core question addressed is whether researchers should use a "real" parser that understands English grammar to dissect these sentences, or if they can rely on "dumb" string manipulation and pattern matching.

Examples of semantic triples extracted from definitions, showing taxonomic (TAX) relationships.

Methodology: Two Philosophies

The authors compared two distinct approaches to extracting semantic relationships:

  1. The Parsing Approach: This "knowledge-heavy" method utilized Sager’s Linguistic String Parser (LSP), involving a 10,000-word lexicon and a massive set of English grammar rules. It attempted to build a complete parse tree for every definition to identify "triples" (Word1, Relation, Word2).
  2. The Text Processing Approach: A "knowledge-light" method using standard UNIX utilities (grep, awk) and custom C code. Instead of full sentence comprehension, it used heuristics (like "Head Finders") to identify the most important words based on their position and common patterns (e.g., "to [verb]..." or "a [noun] that...").

Key Results and Performance Benchmarks

The study found a significant efficiency gap between the two methods. The LSP parser was computationally prohibitive for the era, taking nearly 180 hours of CPU time to process approximately 8,800 definitions. Furthermore, dictionary definitions often omit direct objects (e.g., madden: to make [someone] intensely angry), creating "empty slots" that frequently caused the full parser to fail.

Performance metrics for the LSP parser across different parts of speech.

As shown in the table above, the LSP parser successfully processed 68.65% of the definitions provided. Nouns were the most accessible (77% success), while transitive verbs proved the most difficult (60% success). On average, parsing a single definition took 18.89 seconds per successful parse.

In contrast, the text processing tools achieved a 97.1% accuracy rate for the specific task of identifying the "head" noun in a definition. Development time also favored the simpler approach: it took eight man-months to develop the LSP grammar, compared to only four weeks for the text-processing tools.

Conclusion and Future Directions

The authors noted that the machine-readable version of W7 contained nearly 1,000 typographical errors, which often broke the strict rules of the full parser. They suggest that future systems should not be "all or nothing." Instead of failing entirely on complex sentences, parsers should return partial data.

The paper concludes that the most effective path forward is a hybrid approach: using text processing as a high-speed "first pass" to handle the bulk of the dictionary, and reserving sophisticated parsing as a backup for the most complex and troublesome definitions.

Summary updated

BibTeX
@inproceedings{ahlswede1988parsing,
  author = {Thomas Ahlswede and Martha W. Evens},
  title = {{Parsing vs. Text Processing in the Analysis of Dictionary Definitions}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {1988},
  doi = {10.3115/982023.982050},
}
Figure