An Experiment on Learning Appropriate Selectional Restrictions From a Parsed Corpus

Francesc Ribas FramisCOLING 1994

TL;DR

This paper introduces a method to automatically learn "selectional restrictions" (SRs) – semantic constraints on how words combine – from large text corpora. By leveraging a broad-coverage noun taxonomy and statistical co-occurrence measures, the system can infer these restrictions, offering a more efficient way to build computational lexicons and improve natural language processing systems.

What Problem Does This Paper Address?

Natural language processing (NLP) systems often struggle with understanding the nuances of word meaning and how words typically combine. A key aspect of this is "selectional restrictions" (SRs), which are semantic constraints that a word imposes on its partners. For example, the verb "drink" typically requires an animate subject and a liquid object. Manually defining these SRs for every word in a lexicon is incredibly time-consuming and labor-intensive. Existing methods for automatically acquiring SRs from text either rely on introspection, are limited to specific domains, or are prone to errors. This paper aims to bridge this gap by developing a method to learn SRs at varying levels of abstraction from large, parsed text corpora, making the process more scalable and data-driven.

Methods

The core of the proposed methodology involves learning class-based selectional restrictions. Instead of just looking at word-to-word co-occurrences, the system groups nouns into semantic classes using a broad-coverage noun taxonomy (specifically, WordNet). The process involves three main stages:

  1. Candidate Class Generation: All hyperonyms (parent concepts) of nouns found in the training data are considered potential semantic classes for SRs.
  2. Candidate Evaluation: An "Association Score" is calculated for each verb-class pair in a given syntactic position. This score combines two measures:
  3. Mutual Information (I(v; c/s)): Measures the statistical strength of the association between a verb (v), a syntactic position (s), and a candidate class (c).
  4. Conditional Probability (P(c/v, s)): Favors classes that have more occurrences of nouns, ensuring that the learned restrictions are based on sufficient data.
  5. Best Class Selection: The system iteratively selects the class with the highest Association Score for a given verb and syntactic position. To ensure that the final SRs are mutually disjoint (i.e., not related by hyperonymy), any classes that are hypernyms or hyponyms of the selected best class are removed from the candidate pool for subsequent iterations.

The system extracts co-occurrence triples (verb, syntactic relationship, noun) from a parsed corpus. It uses WordNet for lemmatization and as the semantic taxonomy for clustering nouns.

Datasets

  • Training/Evaluation Corpus: A fragment of the Penn Treebank Corpus, specifically articles from the Wall Street Journal. This corpus contains approximately 880,000 words and 35,000 sentences, which have been tagged and parsed.
  • Lexicons/Taxonomy: WordNet is used for lemmatization and as the semantic taxonomy for clustering nouns into semantic classes.

Key Insights and Ideas

  • Class-based SRs are more robust: Grouping words into semantic classes (like "person" or "legal instrument") allows for more generalizable and robust SRs compared to word-specific restrictions, especially when dealing with limited data for individual words.
  • Leveraging a broad taxonomy is key: Using a comprehensive taxonomy like WordNet provides a rich set of candidate semantic classes, enabling the discovery of SRs at various levels of abstraction.
  • The Association Score balances strength and frequency: The proposed Association Score effectively balances the statistical strength of a co-occurrence with the frequency of the semantic class, leading to more reliable SRs.
  • Iterative disjoint class selection is crucial: The method's ability to iteratively select the best class and then filter out related (hyper/hyponym) classes ensures that the learned SRs are distinct and non-redundant.
  • Parsed corpora are invaluable: The paper highlights that using syntactically analyzed corpora significantly improves the accuracy and representativeness of extracted co-occurrence data, despite the cost of producing such corpora.

Key Results

  • The system successfully extracted 190,766 co-occurrence triples from the corpus.
  • After lemmatization, 93.1% of the triples were correctly mapped to their lemma forms.
  • In a testing sample of 2,658 examples, 89.2% of the extracted triples were considered correctly extracted and lemmatized.
  • Precision: 79.2% of triples appearing in syntactic positions with acquired SRs effectively fulfilled one of those SRs.
  • Recall: 75.7% of triples fulfilled one of the SRs acquired for their corresponding syntactic positions.
  • Manual diagnostics revealed that while many acquired classes were correct (e.g., "Ok" diagnostic), a significant portion (73.3%) were influenced by erroneous senses or noise, though these tended to be smaller in size.

Limitations and Future Work

  • WordNet's coverage of proper nouns: The paper notes that WordNet's information on proper nouns is limited, requiring the authors to assign four predefined classes for them.
  • Ambiguity in verb senses: The SRs acquired can be mixed due to different senses of the same verb appearing in the corpus.
  • Correlation between Assoc and manual diagnostic: The paper observes that classes deemed correct by manual inspection are not always ranked highest by the Association Score.
  • Over-generalization: This can occur when nouns in rival classes are too similar.

Future work suggestions include:

  • Modifying the Association Score to incorporate mutual information between the verb-syntactic position pair and the class.
  • Exploring likelihood ratio tests for estimating probabilities, which may perform better with small counts.
  • Correcting probability estimates by the number of senses of the nouns involved.
  • Collecting more examples by projecting complements onto internal arguments using diathesis sub-categorization rules.
  • Developing methods to handle "implicit negative examples" from PPs attached to other complements to constrain over-generalization.
  • Disambiguating verb senses using acquired SRs to split the set of learned restrictions.

Summary updated

BibTeX
@inproceedings{framis1994experiment,
  author = {Francesc Ribas Framis},
  title = {{An Experiment on Learning Appropriate Selectional Restrictions From a Parsed Corpus}},
  booktitle = {International Conference on Computational Linguistics},
  year = {1994},
}
Figure