Without lexicons, multiword expression identification will never fly: A position statement

Agata Savary, Silvio Cordeiro, Carlos RamischACL 2019

TL;DR

This paper argues that current methods for identifying multiword expressions (MWEs) in text are struggling, especially with unseen data. The authors propose that integrating MWE discovery with syntactic lexicons is crucial for significant progress, moving beyond purely data-driven approaches.

What Problem Does This Paper Address?

Multiword expressions (MWEs) are combinations of words that, as a unit, have a meaning that isn't simply the sum of their individual parts (e.g., "kick the bucket" doesn't literally mean to strike a pail). Identifying these MWEs automatically in text is vital for many natural language processing (NLP) tasks that rely on understanding meaning. However, current MWE identification (MWEI) systems, particularly for general language MWEs (GL-MWEs), perform poorly, especially when encountering MWEs not seen during training. This is a significant bottleneck for applications that need to understand nuanced language.

Methods

The paper doesn't introduce new algorithms but rather presents a position statement based on an analysis of existing MWEI approaches and MWE characteristics. The authors categorize MWEs into sublanguage MWEs (SL-MWEs, like named entities) and general language MWEs (GL-MWEs, like idioms). They analyze properties like proliferation speed, discrepancy from regular word combinations, component similarity, and ambiguity to explain why GL-MWEs are harder to identify. The core proposal is to couple MWEI with MWE discovery, leveraging syntactic MWE lexicons that capture essential morphosyntactic information.

Datasets

The paper references results from the PARSEME shared tasks on automatic identification of verbal MWEs. It also mentions datasets used in named entity recognition (NER) tasks like CoNLL 2002 and 2003, and biomedical NER datasets (JNLPBA, i2b2). No new datasets are introduced.

Key Insights and Ideas

  • GL-MWEs are inherently harder: Unlike SL-MWEs (like named entities) which often have clear surface-level cues (capitalization, trigger words), GL-MWEs are often idiosyncratic at the type level (the abstract concept) but regular at the token level (individual occurrences). This makes them difficult for standard machine learning models that rely on token-level patterns.
  • Unseen data is the Achilles' heel: Current MWEI systems perform drastically worse on unseen MWEs compared to those seen during training. This gap is much larger for GL-MWEs than for SL-MWEs.
  • Lexicons are not dead: Despite the rise of data-driven methods, the authors argue that syntactic MWE lexicons are essential. These lexicons should not aim for complete linguistic coverage but provide minimal, useful morphosyntactic information to complement annotated corpora.
  • Discovery and Identification must be coupled: The paper advocates for a paradigm shift where MWE identification is not done in isolation but is tightly integrated with MWE discovery processes, using these syntactic lexicons as a bridge.

Key Results

The paper primarily analyzes existing results rather than presenting new ones.

  • PARSEME Shared Task 1.1: Top systems for identifying verbal MWEs (GL-MWEs) achieved F1-scores ranging from 0.5 to 0.58. Performance on unseen data was significantly lower, with scores as low as 0.18-0.31 for some systems, representing a 2.5 to 4.6 times worse performance compared to seen data.
  • SL-MWEs (NER): In contrast, NER systems on CoNLL datasets achieved F1-scores up to 0.86, and performance on unseen data remained high (0.81-0.94).
  • Lexical resources help: Studies show that adding automatically discovered lexicons can improve MWEI quality.

Limitations and Future Work

  • Limitations: The authors acknowledge that current MWEI systems struggle with unseen data, especially for GL-MWEs. The performance gap between seen and unseen data is a major limitation. The heterogeneity of MWEs and the difficulty in defining similarity for machine learning are also challenges.
  • Future Work: The authors propose a roadmap:
  • Future shared tasks should include subtasks specifically for unseen data.
  • Develop large-coverage syntactic MWE lexicons that capture lemmas, dependency structures, and morphosyntactic variants.
  • These lexicons should be compatible with standard corpus formats and prioritize rare MWEs.
  • The community should work towards unified multilingual reference datasets combining annotated corpora and NLP-oriented MWE lexicons.

Summary updated

BibTeX
@inproceedings{savary2019without,
  author = {Agata Savary and Silvio Cordeiro and Carlos Ramisch},
  title = {{Without lexicons, multiword expression identification will never fly: A position statement}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
}
Figure