VMWE identification with models trained on GUD (a UDv.2 treebank of Standard Modern Greek)

Stella Markantonatou, Vivian Stamou, Stavros Bompolas, Katerina Anastasopoulou, Irianna Linardaki Vasileiadi, Konstantinos Diamantopoulos, Yannis Kazos, Antonios AnastasopoulosNAACL 2025

TL;DR

This paper introduces UD_Greek-GUD, a new Universal Dependencies treebank for Standard Modern Greek (SMG) that specifically annotates Verbal Multiword Expressions (VMWEs). They present a novel method for integrating VMWE information directly into syntactic dependency relations, enabling models to identify these complex linguistic units.

What Problem Does This Paper Address?

Multiword expressions (MWEs), particularly verbal MWEs (VMWEs), pose a significant challenge for natural language processing due to their idiomatic nature and structural complexity. While research on MWEs in Modern Greek exists, systematic treatments within syntactic parsing frameworks are rare. Existing approaches often rely on symbolic methods. This paper aims to bridge this gap by creating a rich linguistic resource (UD_Greek-GUD) and developing a computational method to improve VMWE identification within syntactic parsing for SMG.

Methods

The core of the paper's contribution lies in its annotation strategy and processing pipeline.

  • UD_Greek-GUD Treebank: This is a new Universal Dependencies (UD) v2 treebank for Standard Modern Greek (SMG). It builds upon the existing UD_Greek-DT (GDT) treebank but adds explicit annotation for Verbal Multiword Expressions (VMWEs) following the PARSEME guidelines. The annotation process involved graduate students over three rounds, with careful attention to morphological and syntactic decisions, including specific guidelines for tokenization, contractions, and functional words.
  • VMWE Integration into Syntactic Layer: Instead of using a separate column for VMWEs, the authors integrate this information directly into the syntactic dependency layer. This is achieved by encoding VMWEs as "sub-relations" appended to existing dependency labels in the DEPREL column of the CoNLL-U format.
  • Preprocessing and Evaluation Pipeline: A custom script, move_mwes.py, is introduced to transfer VMWE annotations from the MISC column (where they are initially stored) to the DEPREL column for model training. This script can also reverse the process. Another script, evaluate_mwes.py, is used for systematic evaluation, comparing model predictions against gold-standard annotations.

Datasets

  • UD_Greek-GUD (GUD): This is the primary dataset introduced in the paper. It comprises 2,530 sentences, combining:
  • 1,807 sentences randomly selected from fiction texts and colloquial SMG sources.
  • 723 sentences specifically annotated for VMWEs, retrieved from the IDION database.
  • UD_Greek-DT (GDT): This existing UD treebank for SMG served as a baseline and was used for initial morphological and syntactic annotations before the GUD VMWE annotations were added.
  • IDION: An open-source web database of SMG VMWEs, used to augment the GUD dataset with VMWE-rich sentences.

Key Insights and Ideas

  • VMWEs as Syntactic Sub-relations: The innovative idea of encoding VMWEs as sub-relations within the standard syntactic dependency structure is a key contribution. This allows existing dependency parsers to directly predict VMWEs without requiring specialized MWE parsers.
  • Impact of VMWE-rich Data: The experiments suggest that training models exclusively on VMWE-rich data (the 723-sentence subset) can lead to the highest per-token precision and recall for VMWE identification. This highlights the importance of specialized data for improving performance on complex linguistic phenomena.
  • Trade-offs in Corpus Size: While larger corpora (like GUD+500) can improve overall VMWE identification (per-unit recall), there might be minor trade-offs in precision compared to smaller, highly specialized datasets.
  • Importance of Token-Level Accuracy: The study indicates that models trained on VMWE-rich data show a slight advantage in correctly predicting token-level VMWE components, which is crucial for accurate MWE recovery.

Key Results

The paper presents quantitative results from training Stanza models on different configurations of the GUD dataset.

  • Syntactic Parsing Performance (Table 1): The models achieve high performance on standard UD metrics (UAS and LAS), with scores generally above 80%. The GUD+723 setting shows slightly better performance than the original GUD corpus alone.
  • VMWE Identification Performance (Table 2):
    • Per-token recall (PTR): The highest PTR (0.827) was achieved with the 723-only training setting.
  • Per-token precision (PTP): The highest PTP (0.88) was also observed in the 723-only setting.
  • Per-unit recall (PUR): The best PUR (0.655) was achieved with the GUD+500 setting, suggesting that larger corpora can improve the identification of complete VMWE units.
  • The GUD+300 setting consistently underperformed, reinforcing the importance of sufficient VMWE-specific training data.

Limitations and Future Work

  • Limitations:
    • The current approach cannot distinguish between nested VMWEs.
  • The model itself does not generate VMWE indexes, requiring manual annotation for newly parsed data when integrating VMWEs back into the MISC column.
  • Per-token evaluation, while useful, doesn't fully capture the nuances of recognizing all lexicalized elements of a VMWE unit.
  • The evaluation methods don't currently differentiate between identification and discovery performance.
  • Future Work:
    • Expand experiments with larger test sets and corpora covering a wider variety of MWE types.
  • Explore encoding VMWE subrelations in the XPOS column to dissociate them from syntactic annotation.
  • Develop more informative evaluation metrics.
  • Address the issue of nested VMWEs and improve the automatic generation of VMWE indexes.
  • Explore evaluation methods that better capture the nuances of VMWE recognition and differentiate between identification and discovery.

Summary updated

BibTeX
@inproceedings{markantonatou2025vmwe,
  author = {Stella Markantonatou and Vivian Stamou and Stavros Bompolas and Katerina Anastasopoulou and Irianna Linardaki Vasileiadi and Konstantinos Diamantopoulos and Yannis Kazos and Antonios Anastasopoulos},
  title = {{VMWE identification with models trained on GUD (a UDv.2 treebank of Standard Modern Greek)}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure