Computer Aided Interpretation of Lexical Cooccurrences

Paola Velardi, Maria Teresa Pazienza

TL;DR

This paper introduces PETRARCA, a system designed to automatically build a semantic lexicon by analyzing word cooccurrences in large text corpora. It bridges the gap between syntactic analysis and semantic interpretation by mapping syntactic patterns to conceptual relations, offering a practical approach to acquiring lexical knowledge for natural language processing.

What Problem Does This Paper Address?

Building comprehensive semantic lexicons for natural language processing (NLP) is a monumental task. Traditional methods rely on manual knowledge engineering, which is time-consuming, expensive, and prone to inconsistencies. The sheer volume of words and their nuanced meanings make manual creation of a truly useful lexicon for broad applications practically impossible. This paper tackles the challenge of lexical semantic knowledge acquisition by proposing an automated method that leverages the vast amount of text data available today. The core problem is how to extract meaningful semantic information from raw text in a way that is both scalable and useful for NLP tasks.

Methods

PETRARCA employs a multi-stage approach to extract and interpret lexical cooccurrences:

  1. Corpus Analysis: It starts with a large corpus of text (in this case, press agency releases on finance and economics) to identify instances of a target word (W).
  2. Syntactic Parsing: A high-coverage morphosyntactic analyzer is used to detect syntactically valid cooccurrences (phrasal patterns) centered around W. This analyzer has a reported 100% coverage of elementary lemmata in the analyzed corpus.
  3. Syntactic to Semantic Mapping: A crucial component is a set of "Syntax-to-Semantic" (SS) rules. These rules interpret the syntactic patterns derived in the previous step and map them to a set of candidate conceptual relations. The paper mentions a catalogue of about 200 such SS rule pairs.
  4. Selectional Restrictions and Type Hierarchies: To resolve ambiguities and select the most appropriate interpretation, PETRARCA uses selectional restriction rules (CR rules) and a hierarchy of conceptual classes. These rules ensure that the semantic interpretation is consistent with the typical properties of the concepts involved. For example, a CR rule might specify that the "participant" relation typically involves a "human entity."
  5. Generalization: The system then generalizes these interpretations to create more abstract semantic descriptions. This involves identifying common patterns across multiple syntactic attachments and deriving more general constraints.

The authors distinguish between "conceptual meaning" (deep sense) and "collocative meaning" (word usage and associations). PETRARCA primarily focuses on acquiring collocative meaning from corpora, which they argue is more objectively observable and thus more amenable to automated acquisition.

Datasets

  • Corpus: The primary dataset used is a large sample of press agency releases on finance and economics. The paper mentions a test performed on about 50 occasional visitors and staff members of the IBM Rome scientific center, who interacted with the system using this corpus. The test was later extended to the Italian Constitution.
  • Lexicons/Knowledge Bases:
    • A morphosyntactic analyzer with a lexicon of 7000 elementary lemmata.
  • A catalogue of patterns/interpretation pairs (SS rules).
  • Selectional restriction rules (CR rules).
  • A hierarchy of conceptual classes and a catalogue associating words to concept types.

The paper does not introduce a new benchmark dataset but rather uses existing corpora for knowledge acquisition.

Key Insights and Ideas

  • Surface Semantics for Acquisition: The paper strongly advocates for "surface meaning" (collocative meaning) as a more practical and automatable approach to semantic lexicon acquisition compared to "deep" conceptual meaning. This is because word associations are observable phenomena in text, whereas deep semantic representations often rely on subjective human-defined primitives.
  • Syntactic Patterns as Semantic Clues: The core idea is that syntactically valid cooccurrences, when interpreted through a set of well-defined rules, can reveal semantic relationships between words. This bridges the gap between syntactic parsing and semantic understanding.
  • The Power of Cooccurrence Data: The availability of large digital text corpora provides an unprecedented opportunity to gather empirical evidence of word usage, which can be leveraged for building lexicons.
  • Semi-Automatic Acquisition is Key: While PETRARCA aims for automation, the authors acknowledge that a manual step (checking the newly derived entry) is necessary, highlighting the ongoing need for human oversight in knowledge acquisition.

Key Results

The paper reports on the effectiveness of a semantic lexicon built using a similar surface meaning approach (limited to about 1000 manually entered definitions) in a previous system called DANTE. This system was able to answer more than 90% of questions concerning previously analyzed sentences from press agency releases, demonstrating that surface semantics can capture a significant portion of human understanding of text. While specific quantitative results for PETRARCA itself are not detailed in terms of overall lexicon coverage or accuracy, the system's ability to process raw language expressions and perform classification and encoding is highlighted as a significant achievement.

Limitations and Future Work

  • Idiomatic Expressions: PETRARCA, in its current version, does not automatically interpret idiomatic expressions. Users must provide the correct interpretation.
  • Error Handling: The system does not currently handle errors or incorrect interpretations. Error correction and refinement of the knowledge base are performed by the programmer.
  • Metaphors: While syntactic evidence can detect metaphors, the system does not provide automatic interpretation of them.
  • Type Hierarchies: The definition and acquisition of type hierarchies for conceptual classes are ongoing research areas. The paper discusses challenges in creating consistent and comprehensive hierarchies and explores methods for semi-automatic word clustering.
  • Generalization Policies: The system is studying different policies for generalizing case descriptions, including the exploitation of counterexamples.

Future work includes improving the interpretation of idiomatic expressions, developing robust error correction mechanisms, and advancing the automatic acquisition and organization of type hierarchies.

Summary updated

BibTeX
@inproceedings{velardi1989computer,
  author = {Paola Velardi and Maria Teresa Pazienza},
  title = {{Computer Aided Interpretation of Lexical Cooccurrences}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {1989},
}
Figure