Mumpitz at PARSEME Shared Task 2018: A Bidirectional LSTM for the Identification of Verbal Multiword Expressions

Rafael Ehren, Timm Lichte, Younes SamihCOLING 2018

TL;DR

This paper introduces Mumpitz, a system for automatically identifying verbal multiword expressions (VMWEs) using a Bidirectional Recurrent Neural Network (BRNN) with Long Short-Term Memory (LSTM) units. The system combines this neural approach with a heuristic that leverages dependency information to distinguish between different VMWEs, achieving competitive results in the PARSEME Shared Task.

What Problem Does This Paper Address?

Natural language is full of expressions where the meaning isn't simply the sum of its parts – these are called multiword expressions (MWEs). Verbal multiword expressions (VMWEs), like "give up" or "look into," are particularly challenging for computers to identify due to their idiomatic nature, potential for discontinuity (e.g., "give it up"), and ambiguity. The PARSEME Shared Task aims to tackle this by focusing on the identification of VMWEs in running text across multiple languages. This paper addresses the challenge of accurately detecting these complex linguistic units in a multilingual setting.

Methods

Mumpitz employs a Bidirectional Recurrent Neural Network (BRNN) with Long Short-Term Memory (LSTM) units. BRNNs are effective for sequence processing as they read input in both forward and backward directions, capturing context from both sides of a word. LSTMs are a type of RNN specifically designed to handle long-range dependencies, which is crucial for VMWEs where components might be separated by other words.

Beyond the core neural network, Mumpitz incorporates a heuristic that utilizes dependency information from the PARSEME corpus. This heuristic helps to differentiate between various VMWEs within a sentence, a task the base BRNN alone doesn't explicitly handle. For the "closed track" of the shared task, word embeddings were randomly initialized. In contrast, for the "open track," pre-trained word embeddings were used to potentially boost performance.

Datasets

The primary dataset used is the PARSEME corpus, which provides annotated data for VMWE identification across 20 languages. The authors submitted results for 7 languages in the closed track and 1 language in the open track. For training word embeddings in the open track, they utilized the DECOW16 corpus, a large German web corpus.

Key Insights and Ideas

  • LSTM's strength for VMWEs: The paper highlights how LSTMs are well-suited for VMWE identification due to their ability to remember relevant information over long distances, which is essential for discontinuous VMWEs.
  • Dependency information is key: The heuristic leveraging dependency relations proved valuable in distinguishing VMWEs, suggesting that syntactic structure plays a significant role in their identification.
  • Pre-trained embeddings make a difference: Using pre-trained word embeddings (specifically self-trained ones on DECOW16) significantly improved performance in the open track compared to randomly initialized embeddings, demonstrating the value of external linguistic knowledge.
  • Feature selection matters: The choice of features (lemma, language-specific POS-tag, and dependency relation) was optimized on German data and then applied to other languages, showing the impact of feature engineering.

Key Results

In the closed track, Mumpitz achieved a 5th place ranking (out of 11 systems) for MWE-based evaluation and 1st place for token-based evaluation on the German test set. For the open track, Mumpitz-preinit (the version with pre-trained embeddings) ranked 4th for MWE-based and 1st for token-based evaluation on German. The use of pre-trained embeddings led to a substantial increase in precision and an overall improvement in the F1-score for both MWE-based and token-based evaluations.

Limitations and Future Work

The authors acknowledge that their heuristic for differentiating VMWEs is deliberately simple and could be improved. A significant limitation is that Mumpitz does not categorize VMWEs into specific types (like VPCs, LVCs, etc.), which is a capability of some other systems. Future work could involve:

  • Developing language-specific feature sets instead of a single optimized set.
  • Integrating VMWE type classification directly into the classifier rather than relying on a post-hoc heuristic.
  • Exploring character-level embeddings as additional features to potentially enhance performance.

Summary updated

BibTeX
@inproceedings{ehren2018mumpitz,
  author = {Rafael Ehren and Timm Lichte and Younes Samih},
  title = {{Mumpitz at PARSEME Shared Task 2018: A Bidirectional LSTM for the Identification of Verbal Multiword Expressions}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2018},
}
Figure