TL;DR
This paper proposes using "selectional preferences" of verbs to automatically detect non-compositional verb-object combinations, which are phrases where the meaning isn't simply the sum of its parts (like "kick the bucket"). They found that models focusing on "types" of arguments, rather than just individual word occurrences, and especially those using automatically acquired thesauri, significantly outperform previous methods.
What Problem Does This Paper Address?
Many common phrases in language, like "kick the bucket" or "spill the beans," are "non-compositional." This means their meaning cannot be directly inferred from the individual words. Understanding and identifying these phrases is crucial for applications like machine translation, information extraction, and paraphrasing, as they often need to be treated as single units. Previous work has explored various statistical and distributional methods to detect this non-compositionality, but there's a need for more robust and accurate approaches. This paper specifically targets verb-object combinations and proposes using a verb's "selectional preferences" – what kinds of objects a verb typically "prefers" – as a signal for non-compositionality. The hypothesis is that if an object is an "atypical" choice for a verb, the combination is more likely to be non-compositional.
Methods
The core of this paper is the development and evaluation of three models for acquiring selectional preferences:
- TCM (Tree Cut Model): This is a reimplementation of a previous method (Li and Abe, 1998) that uses WordNet to define noun classes. It selects classes based on the Minimum Description Length (MDL) principle, aiming for a balance between model simplicity and data explanation. Crucially, it uses tokens (individual word occurrences) to select classes.
- WNPROTO (WordNet Prototypical Classes): This is a novel "type-based" approach using WordNet. Instead of focusing on token counts, it prioritizes classes that cover a higher proportion of distinct argument types from the training data. This aims to reduce the influence of noisy or infrequent arguments and focus on more representative semantic categories.
- DSPROTO (Distributional Thesaurus Prototypical Classes): This is another novel "type-based" approach, but it uses an automatically acquired distributional thesaurus (based on Lin, 1998) instead of WordNet. Similar to WNPROTO, it selects classes based on the number of argument types they cover, employing a best-first search strategy to identify the most informative classes.
All models are trained on verb-object pairs extracted from the British National Corpus (BNC). The selectional preferences are then used to calculate a probability score for a given verb-object pair. A lower probability suggests a more atypical object for the verb, and thus a higher likelihood of non-compositionality.
Datasets
- British National Corpus (BNC): Used for acquiring verb-object pairs and for training the selectional preference models. The RASP parser was used to extract grammatical relations.
- Venkatapathy and Joshi (2005) dataset: This dataset of 765 verb-object collocations, with human judgments of compositionality, was used for evaluation. The authors focused on a subset of 638 pairs involving common nouns in the object relation.
Key Insights and Ideas
- Type-based selectional preferences are superior: Focusing on the types of arguments a verb prefers (e.g., "vehicles" for "drive") is more effective than simply counting individual word occurrences (tokens). This helps mitigate noise from frequent but potentially polysemous words.
- Distributional thesauri offer a powerful alternative: The DSPROTO model, using an automatically generated thesaurus, achieved the best results. This is significant because it means the approach can be applied to languages or domains where a comprehensive, hand-crafted resource like WordNet might not exist.
- Selectional preferences are a strong indicator of non-compositionality: The probability scores derived from these preferences show a highly significant correlation with human judgments of compositionality, outperforming many individual features used in prior work.
- Combining preferences with other features is optimal: While selectional preferences perform well on their own, combining them with other features (like frequency and mutual information) using SVMs leads to the best overall results.
Key Results
The paper reports Spearman's rank-correlation coefficients (rs) for their models against the gold-standard human judgments.
- The TCM model achieved an rs of 0.090.
- The WNPROTO model achieved an rs of 0.223.
- The DSPROTO model achieved the highest correlation among the preference models, with an rs of 0.398.
When combined with other features using SVMs:
- A combination of previous features (f1, f2, f3, f7) achieved an rs of 0.419.
- The DSPROTO model combined with these features (DSPROTO f1,2,3,7) achieved the best result with an rs of 0.454.
This demonstrates that the DSPROTO model, in particular, provides a strong signal for compositionality, and its integration with other features yields optimal performance.
Limitations and Future Work
- Parser noise: The authors acknowledge that systematic parsing errors (e.g., misclassifying adverbials as objects) can impact the selectional preference models.
- Limited scope: The current work focuses on verb-object combinations. Future work could explore other types of multiword expressions, such as verb-particle constructions.
- Integration with other features: While combining selectional preferences with other features improved results, the authors suggest further exploration of which combinations are most effective. They also propose combining preferences with measures of syntactic fixedness.
- Application to other tasks: The authors believe their selectional preference models could be valuable for other NLP tasks, such as diathesis alternation detection and contrastive analysis with existing WordNet models.
- Data sources: They suggest investigating the use of larger, raw text corpora like Google's Web 1T corpus, as web counts have shown promise in similar tasks.
Summary updated