Towards the semantic annotation of SR-ELEXIS corpus: Insights into Multiword Expressions and Named Entities

Cvetana Krstev, Ranka Stanković, Aleksandra M. Marković, Teodora Sofija MihajlovCOLING 2024

TL;DR

This paper introduces the SR-ELEXIS corpus, a Serbian extension to the multilingual ELEXIS corpus, focusing on the semantic annotation of multiword expressions (MWEs) and named entities (NEs). By integrating Serbian data and developing annotation strategies, the work aims to improve NLP tasks like word sense disambiguation for low-resource languages and explore Linked Open Data approaches for lexical resources.

What Problem Does This Paper Address?

High-quality, openly accessible annotated corpora are crucial for advancing NLP tasks like word sense disambiguation (WSD) and natural language understanding (NLU). However, such resources are scarce for many languages, particularly those with limited resources like Serbian. This scarcity stems from the lack of readily available sense inventories and the significant cost and manpower required for manual annotation, often needing expert linguists. The paper addresses this by extending the existing multilingual ELEXIS corpus with Serbian data, focusing on the complex annotation of MWEs and NEs, and proposing methods for their integration into a sense repository and future LLOD representations.

Methods

The paper details a multi-stage process for annotating the Serbian ELEXIS-SR corpus:

  • Automatic Translation: Sentences from the English ELEXIS-WSD corpus were automatically translated into Serbian using Google Translate to create an initial Serbian dataset.
  • Manual Correction and Refinement: This automatic translation underwent rigorous manual checking by native Serbian speakers to correct errors, resolve pronoun references, and ensure accurate phonetic transcription of names. This was a time-consuming but essential step for data quality.
  • Automatic Annotation Pipeline: The corrected Serbian sentences were then processed through an automatic pipeline including tokenization, lemmatization, and POS/UPOS tagging.
  • MWE and NE Annotation: Specific tools and resources were employed for MWE and NE annotation:
  • An e-dictionary of non-verbal MWEs, built on principles similar to Serbian WordNet, was used.
  • A Named Entity Recognition (NER) system based on e-dictionaries and rules was applied.
  • A system for recognizing verbal MWEs, leveraging the PARSEME corpus, was utilized.
  • A system for adjectival and verbal similes was also employed.
  • Sense Repository Integration: The Serbian sense repository is being built based on the Serbian WordNet (SrpWN), aligning it with the English ELEXIS sense repository and Princeton WordNet.
  • LLOD Exploration: The paper proposes using Linked Open Data (LLOD) standards like Ontolex-lemon and DMLex for representing MWE lexicon entries and linking them to corpus occurrences, facilitating cross-lingual analysis.

Datasets

  • ELEXIS-WSD Corpus: A parallel sense-annotated corpus for 10 languages, serving as the foundation.
  • SR-ELEXIS Corpus: The newly introduced Serbian extension of the ELEXIS-WSD corpus, created through automatic translation and manual correction of English sentences.
  • Serbian WordNet (SrpWN): Used as the basis for the Serbian sense repository.
  • PARSEME Corpus: Utilized for annotating verbal MWEs in the Serbian dataset.
  • Wikipedia Texts: Mentioned as a source for other sense-annotated corpora (e.g., OneSec).

Key Insights and Ideas

  • MWEs and NEs are Crucial for Low-Resource Languages: The paper highlights that annotating MWEs and NEs is as important as annotating single words for building robust NLP systems, especially for languages like Serbian.
  • Automatic Translation is a Necessary First Step, but Imperfect: While automatic translation speeds up corpus creation, manual verification is indispensable for achieving high-quality, linguistically accurate data, particularly for complex linguistic phenomena like MWEs.
  • Cross-Lingual Comparison Reveals Annotation Discrepancies: Comparing MWE and NE annotations across languages in the ELEXIS corpus shows significant differences, likely due to varying annotation resources and methodologies.
  • LLOD for Lexical Resources is Promising: The exploration of Ontolex-lemon and DMLex for representing MWEs and linking them to corpora offers a standardized and interoperable way to share and utilize lexical knowledge.
  • Distinguishing MWEs from NEs is a Challenge: The paper acknowledges the ongoing research needed to establish clear guidelines for differentiating MWEs from NEs, especially in the context of Serbian.

Key Results

  • The SR-ELEXIS corpus is being developed, with the Serbian sentence set undergoing automatic translation and initial annotation stages.
  • Initial results show varying numbers of MWEs and NEs across the 10 languages in the ELEXIS-WSD repository, indicating differences in resource availability and annotation efforts.
  • For the Serbian dataset, 2,024 sentences were processed, yielding 2,006 NE occurrences and 529 non-verbal MWE annotations.
  • A significant portion of MWEs found in the Serbian corpus (39%) were also found in the Serbian WordNet.
  • The paper presents a detailed pipeline for preparing and annotating the Serbian dataset, with many tasks completed or in progress.

Limitations and Future Work

  • Limitations:
    • The automatic translation process, while necessary, required extensive manual correction.
  • The initial annotation of NEs was not systematic across all languages, with some datasets having no NE annotations.
  • The comparison of MWEs and NEs across languages was initially based on automatic translation, which was not always appropriate.
  • The accuracy of MWE/NE recognition, recall, and precision is still to be fully determined.
  • Some MWEs and NEs found in the Serbian corpus were not annotated due to their absence in the used lexical resources.
  • Future Work:
    • Completing the annotation of MWEs and NEs in SR-ELEXIS.
  • Developing a LLOD dictionary for MWEs.
  • Syntactic annotation of the Serbian dataset.
  • Coordinating work with other research groups involved in ELEXIS, PARSEME, UD, and UniDive.
  • Publishing various editions of the SR-ELEXIS corpus, including detailed NE annotation.
  • Developing precise guidelines for distinguishing MWEs from NEs in Serbian.
  • Conducting comparative analyses of MWEs and NEs in the ELEXIS multilingual set from linguistic and NLP perspectives.
  • Associating MWE lexicon entries with their occurrences in the corpus using LLOD.

Summary updated

BibTeX
@inproceedings{krstev2024towards,
  author = {Cvetana Krstev and Ranka Stanković and Aleksandra M. Marković and Teodora Sofija Mihajlov},
  title = {{Towards the semantic annotation of SR-ELEXIS corpus: Insights into Multiword Expressions and Named Entities}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2024},
}
Figure