TL;DR
This paper tackles the challenge of automatically measuring how "compositional" verb-noun collocations are (e.g., "drink coffee" vs. "kick the bucket"). They introduce novel features and integrate them with existing ones using a Support Vector Machine (SVM) ranking approach, achieving significantly better correlation with human judgments than individual features alone. This work is crucial for Natural Language Processing (NLP) applications that need to understand the idiomatic nature of language.
What Problem Does This Paper Address?
Multi-word expressions (MWEs), like "kick the bucket" (meaning "die") or "drink coffee" (meaning literally to consume coffee), pose a challenge for NLP systems. Their meaning isn't always a simple sum of their parts; they can be idiomatic or "non-compositional." Accurately measuring the degree of compositionality for these expressions is vital for tasks like machine translation, where idiomatic phrases need special handling. While various statistical measures exist to identify collocations, there's a need for a robust way to quantify their relative compositionality on a spectrum, rather than a binary classification. This paper focuses specifically on verb-noun (V-N) collocations and aims to improve upon existing methods by integrating a richer set of features.
Methods
The authors propose a two-pronged approach:
- Novel Feature Engineering: They introduce new features to capture compositionality, categorized into:
- Collocation-based features: These consider the entire collocation as a unit. Examples include:
- Distributed Frequency of Object (d): Measures how many different verbs an object typically co-occurs with. If an object appears with very few verbs, it suggests an idiomatic usage.
- Distributed Frequency of Object using Verb Information (ρ): An extension of 'd' that accounts for the similarity between verbs co-occurring with the same object, aiming to differentiate between similar and dissimilar verb usages.
- Context-based features: These leverage Latent Semantic Analysis (LSA) to represent words and collocations in a vector space. Key features include:
- Dissimilarity of the collocation with its constituent verb (ι): Measures how semantically different the collocation is from its verb. High dissimilarity suggests non-compositionality.
- Similarity of the collocation to the verb-form of the object (ψ): Measures how similar the collocation is to the verbal form of its object (e.g., "give a smile" vs. "to smile"). High similarity suggests the verb is acting as a support verb, indicating non-compositionality.
- SVM-based Ranking: They integrate these novel features along with traditional ones (like Frequency and Point-wise Mutual Information) into a Support Vector Machine (SVM) ranking function. This function learns to weigh the features and produce a ranked list of collocations based on their compositionality, aiming to mimic human judgments.
Datasets
The primary dataset used is the British National Corpus (BNC), a large collection of English text. From the BNC, they extracted V-N collocations. For evaluation, they created a dataset of 800 V-N collocations. These were manually annotated by two human judges on a scale of 1 to 6, where 1 is completely non-compositional and 6 is completely compositional. They also utilized WordNet and other lexical resources for feature extraction.
Key Insights and Ideas
- Compositionality is a Continuum: The paper reinforces the idea that compositionality isn't a binary property but exists on a spectrum, necessitating ranking-based approaches.
- Context Matters: Context-based features, particularly the similarity to the verb-form of the object (ψ), proved to be strong indicators of compositionality, outperforming many traditional collocation-based features individually.
- Feature Integration is Key: Combining multiple features, especially through a machine learning model like SVM, significantly boosts performance compared to relying on any single feature. This suggests that different features capture complementary aspects of compositionality.
- Human Judgment as Gold Standard: The ultimate goal is to align with human intuition, and the SVM approach, by learning from human-annotated data, effectively bridges the gap between statistical measures and human perception.
Key Results
- The proposed contextual feature, 'Similarity of the collocation to the verb-form of the object' (ψ), showed a significantly higher correlation with human rankings than other individual features.
- Other features like 'Least mutual information difference with similar collocations' (μ) and 'Distributed frequency of object using the verb information' (ρ) also performed well.
- The SVM-based ranking function, integrating all features, achieved a significantly better correlation with human judgments (Pearson's Rank-Order Correlation Coefficient, r_s = 0.448) than any individual feature alone. For instance, the best individual feature correlation was 0.300.
- Figure 1 demonstrates that as more features are integrated into the SVM, the correlation with human rankings generally improves, highlighting the benefit of a comprehensive feature set.
Limitations and Future Work
- Limitations: The authors acknowledge that context-based features, while performing well individually, did not contribute as much to the SVM-based ranking function as expected, suggesting potential areas for improvement in how these features are integrated or represented.
- Future Work: The authors plan to evaluate their techniques in applications like Machine Translation. They also intend to extend their approach to other types of MWEs and to collocations in other languages, with work on Hindi already in progress.
Summary updated