Collocation Classification with Unsupervised Relation Vectors

Luis Espinosa Anke, Steven Schockaert, Leo WannerACL 2019 · OpenReview

Abstract

Luis Espinosa Anke, Steven Schockaert, Leo Wanner. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.

2019.

This paper explores the intersection of distributional semantics and lexicography, investigating how well modern NLP models can categorize collocations—semi-fixed word pairings like "heavy rain" or "make a decision." The authors introduce LexFunC, a novel dataset for classifying collocations based on semantic functions, and demonstrate that while unsupervised relation vectors boost performance, collocations remain significantly more challenging to classify than standard semantic relations.

Introduction

In NLP, relation classification typically focuses on standard semantic links, such as hypernymy (e.g., "dog" is an "animal") or capital-city relations. However, collocations—idiosyncratic, semi-compositional pairings—represent a significant gap in current modeling. Unlike logical semantic relations, collocations are often language-specific and syntactically bound (e.g., "taking" a nap in English vs. "throwing" a nap in other languages). This study evaluates whether word embeddings and relation vectors can capture the subtle semantic functions underlying these pairings.

The LexFunC Dataset

The primary contribution of this work is LexFunC, a dataset containing 4,914 collocations categorized into 10 "Lexical Functions" (LFs) derived from Meaning-Text Theory. These functions describe the semantic role of a collocate relative to its base.

Table 1: Lexical Functions (LFs) in the LexFunC dataset, including definitions, examples, and frequencies.

As shown in the table, the dataset is significantly skewed, with the magn (intensity) category being the most frequent, while others like mult (collections) are relatively rare.

Methodology

The researchers framed collocation classification as a multi-class supervised learning problem using a Linear SVM. They compared several input feature configurations:

  1. Vector Arithmetic Baselines: Standard operations on GloVe embeddings, including vector difference (diff), concatenation (concat), sum, and multiplication.
  2. Unsupervised Relation Vectors: Utilizing the SeVeN model to represent the context between word pairs in a large corpus (Wikipedia).
  3. rvAvg6: A high-dimensional vector capturing words appearing before, between, and after the target pair.
  4. rvAE: A 300d compressed vector produced via a conditional autoencoder.

Results and Analysis

The experimental results indicate that collocations are a "boss level" challenge for relational models. Performance on LexFunC was consistently lower than on DiffVec, a dataset of standard semantic relations.

Table 2: Performance comparison (Precision, Recall, F1, and Accuracy) across different vector operations and relation vector enhancements.

Key findings include:

  • Concatenation Superiority: Unlike many relation tasks where vector subtraction is preferred, concatenation performed best for collocations. This suggests that preserving the identity of both words is crucial for modeling idiosyncratic pairings.
  • Contextual Boost: Adding unsupervised relation vectors (rvAvg6) provided a performance increase of nearly 10% in some instances, highlighting the importance of the syntactic "glue" between words.
  • Performance Gap: The best model achieved an F1 score of 67.70% on LexFunC, compared to 88.09% on DiffVec, confirming that lexical functions are more subtle and difficult to distinguish than standard relations.

Conclusion

While unsupervised relation vectors significantly improve the classification of collocations, the task remains difficult due to data imbalance and the idiosyncratic nature of lexical functions. Future work may involve cross-lingual transfer and the use of pairwise correlation metrics to refine training data and address lexical memorization.

Summary updated

BibTeX
@inproceedings{anke2019collocation,
  author = {Luis Espinosa Anke and Steven Schockaert and Leo Wanner},
  title = {{Collocation Classification with Unsupervised Relation Vectors}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  url = {https://openreview.net/forum?id=IeXfX40158w},
}
Figure