A re-examination of lexical association measures

Huu Hung Hoang, Su Nam Kim, Min-Yen KanMWE@IJCNLP 2009 · OpenReview

Abstract: Hung Huu Hoang, Su Nam Kim, Min-Yen Kan. Proceedings of the Workshop on Multiword Expressions: Identification, Interpretation, Disambiguation and Applications (MWE 2009).

2009.

Introduction

In Natural Language Processing (NLP), identifying Multiword Expressions (MWEs)—phrases like "kick the bucket" (idioms), "put on" (verb-particle constructions), or "make a decision" (light verb constructions)—is a fundamental yet notoriously difficult task. Researchers traditionally rely on Lexical Association Measures (AMs), mathematical formulas that calculate the statistical "connectedness" of word pairs based on their co-occurrence frequencies.

However, the field has become cluttered with dozens of competing formulas, ranging from Pointwise Mutual Information (PMI) to complex statistical hypothesis tests. This paper provides a necessary "spring cleaning" of these tools by introducing the concept of "rank equivalence" and proposing targeted normalization techniques to improve extraction accuracy.

Methodology: Navigating the Cluttered Landscape

The authors began by evaluating 82 different AMs. To bring order to this variety, they categorized measures into two broad classes:

  1. Class I (Institutionalization): Measures based on frequency statistics that determine how much a phrase acts as a single unit.
  2. Class II (Non-compositionality): Measures based on the diversity or similarity of the contexts in which the words appear.
A subset of the lexical association measures and their formulas, illustrating the mathematical complexity and variety in the field.

Beyond classification, the authors addressed the bias inherent in many popular measures. Standard measures like PMI often favor low-frequency words, while others fail when one component (like the light verb "do" or "take") is extremely common. To combat this, they proposed modified versions of PMI and Mutual Information (MI) that incorporate normalization factors ($NF$) and penalization terms tailored to the marginal frequencies of the constituent words.

Key Insights: Rank Equivalence

One of the paper's most significant contributions is the formal definition of rank equivalence. Two different mathematical formulas are considered rank equivalent if they always produce the same ranked list of candidates for any given dataset. This insight allows researchers to bypass complex mathematical derivations in favor of simpler, equivalent formulas without losing any discriminative power.

The authors identified five distinct groups of rank-equivalent measures, significantly simplifying the selection process for researchers.

As shown in the table above, 15 complex AMs were successfully reduced to just five representative groups. For instance, the Odds Ratio, Yule’s ω, and Yule’s Q were found to be rank equivalent, meaning they are essentially the same tool for the purpose of ranking MWE candidates.

Performance and Results

The study utilized the Wall Street Journal section of the Penn Treebank to evaluate these measures against gold-standard datasets for Verb Particle Constructions (VPCs) and Light Verb Constructions (LVCs). The results demonstrated that context-based (Class II) measures performed poorly due to data sparsity in the 1-million-word corpus.

In contrast, the proposed normalization techniques yielded dramatic improvements in Average Precision (AP):

  • Normalized PMI: Improved AP on a mixed MWE task from 51.5% to 58.8%.
  • Normalized MI: Standard MI is typically poor for LVC detection (28.9% AP), but after applying the authors' normalization, it reached 58.3% AP, becoming the top-performing measure for that category.

The authors also discovered that different types of MWEs require different strategies: VPCs benefit from a linear combination of marginal frequencies, while LVCs are better handled using a product-based penalization.

Conclusion

By simplifying the landscape of association measures through rank equivalence and providing a framework for normalization, this work offers a more manageable and effective toolkit for MWE extraction. While the study was limited to a 1-million-word corpus, it provides a robust foundation for future research on larger, web-scale datasets where the behavior of these measures may further evolve.

Summary updated

BibTeX
@inproceedings{hoang2009re,
  author = {Huu Hung Hoang and Su Nam Kim and Min-Yen Kan},
  title = {{A re-examination of lexical association measures}},
  booktitle = {MWE@IJCNLP},
  year = {2009},
  url = {https://openreview.net/forum?id=57aUhw-V6_h},
}

Loading related content...

Figure