Acquisition of Verb Entailment from Text

Viktor PekarNAACL 2006

TL;DR

This paper introduces a novel method for automatically discovering verb entailment relations (e.g., "buy" entails "own") directly from text. By leveraging the idea that these relations manifest in the regular co-occurrence of verbs within locally coherent text segments, the approach achieves promising results, outperforming existing lexical resources and approaching human-level performance.

What Problem Does This Paper Address?

Understanding how events and states relate to each other is fundamental to commonsense reasoning. Verb entailment captures this by identifying directional relationships where one verb implies another (e.g., if someone "buys" an object, it necessarily "belongs" to them). While valuable for tasks like Question Answering and Information Extraction, automatically acquiring these asymmetric relations from text is challenging. Previous methods often rely on specific lexico-syntactic patterns, limiting their coverage, or struggle to distinguish entailment from mere paraphrases. This paper aims to develop a more robust and general method for discovering verb entailment that can handle varied argument structures and capture a wider range of entailment types.

Methods

The proposed method operates in three main steps:

  1. Identifying Discourse-Related Clauses: The system identifies pairs of clauses that are likely related in local discourse. This is achieved by combining several surface cues:
  2. Textual Proximity: Clauses must appear within a certain window of each other.
  3. Paragraph Boundaries: Clauses are expected to be within the same paragraph.
  4. Common Event Participant (Anchor): The clauses must share a common noun lemma as an argument (subject, object, or prepositional object), excluding pronouns to avoid noise.
  5. Creating Templates: From identified discourse-related clauses, pairs of syntactic templates are created. Each template consists of a verb and a label indicating the syntactic role of the anchor relative to the verb. For instance, "Mary bought a house. The house belongs to Mary." yields templates like buy(obj:X) and belong(subj:X). To increase data usability, passive constructions are converted to active, and coordinated phrases are decomposed.
  6. Measuring Asymmetric Association: The core of the method involves scoring the plausibility of entailment between verb pairs. This is done by quantifying the "preference" of a premise verb for a consequence verb using a measure derived from Kullback-Leibler Divergence. The direction with the higher association score is taken as the likely direction of entailment.

Datasets

  • Training/Acquisition: The method was initially developed and tested using the BLLIP corpus (texts from the Wall Street Journal, 1987-89).
  • Evaluation: The evaluation was performed on entailment templates acquired from the British National Corpus (BNC).

Key Insights and Ideas

  • Discourse Structure as a Signal: The central assumption that verb entailment relations are reflected in the regular co-occurrence of verbs within locally coherent text segments is a powerful insight. This moves beyond simple pattern matching to leverage deeper linguistic structure.
  • The Importance of Anchors: Requiring a common event participant (anchor) significantly improves the accuracy of identifying discourse-related clauses and, consequently, verb entailment. This suggests that shared entities are crucial for linking events semantically.
  • Window Size Matters: The optimal window size for identifying related clauses is relatively small (around 7 clauses), indicating that entailment is often expressed in very close proximity. Paragraph boundaries also play a role, suggesting that coherent text segments often align with paragraph structure.

Key Results

  • The proposed method achieved an accuracy of 0.735 on a test set of 102 items, outperforming both the DIRT database (0.647) and WordNet 2.0 (0.38).
  • The system's performance was significantly better than random baseline (0.2).
  • Human evaluation showed that the system's performance was close to that of human judges (difference of ~0.10), with a Kappa statistic of 0.725 for inter-judge agreement.
  • The common anchor criterion was found to be highly effective, leading to significantly better accuracy compared to models without it.

Limitations and Future Work

  • Coreference Resolution: The authors acknowledge that the current model does not account for coreference resolution (nominal and pronominal). They suggest that incorporating coreference techniques could significantly improve the model by better identifying common event participants across text.
  • Limited Scope of Anchor: The current anchor identification is limited to subject, object, and prepositional object. Expanding this could potentially capture more relations.
  • Data Sparsity: While the method is designed to be general, performance can still be affected by the availability of sufficient co-occurrence data for verb pairs.
  • Future Work: The authors propose extending the algorithm with coreference resolution and exploring other linguistic cues for identifying discourse relations.

Summary updated

BibTeX
@inproceedings{pekar2006acquisition,
  author = {Viktor Pekar},
  title = {{Acquisition of Verb Entailment from Text}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2006},
}
Figure