Extended and Enhanced Polish Dependency Bank in Universal Dependencies Format

Alina WróblewskaEMNLP 2018

TL;DR

This paper introduces PDBUD, the largest Polish dependency bank in the Universal Dependencies format, significantly expanding previous resources with over 22,000 trees. This enhanced dataset, featuring richer annotations and corrected data, is crucial for training high-quality dependency parsers for Polish, a morphologically rich language with flexible word order.

What Problem Does This Paper Address?

Developing robust Natural Language Processing (NLP) tools, especially for syntactic parsing, heavily relies on large, high-quality annotated datasets. For morphologically rich languages like Polish, with its relatively free word order, creating such datasets is particularly challenging and time-consuming. Existing Polish dependency treebanks, while valuable, have limitations in size or annotation richness. This paper addresses the need for a more comprehensive and detailed Polish dependency treebank that adheres to the Universal Dependencies (UD) standard, enabling better machine learning models for Polish NLP tasks.

Methods

The core of this work is the creation of the PDBUD dataset. It builds upon existing Polish UD treebanks (PL-SZ) and a new version of the Polish Dependency Bank (PDB). The process involved:

  • Correction and Enhancement: Existing PL-SZ trees were checked and corrected.
  • Automatic Conversion: A significant portion of trees were automatically converted from the new PDB. This conversion required modifications to the UD annotation schema to handle specific Polish linguistic phenomena not present in simpler datasets.
  • Extended Edges: PDBUD introduces "enhanced edges" to capture shared dependents and governors in coordinated structures, providing a more nuanced representation of sentence syntax.
  • Semantic Roles: Semantic roles for some dependents were added, going beyond the standard UD annotation scheme.
  • New UD Subtypes: Several new UD relation subtypes were defined to better represent Polish linguistic features, such as obl:cmpr for comparative constructions, aux:clitic for mobile inflections, and aux:cnd for conditional particles.

Datasets

  • PDBUD (Polish Dependency Bank in Universal Dependencies format): The primary contribution of this paper. It contains 22,208 trees and 351,715 tokens.
  • PL-SZ (Polish UD treebank): An existing Polish UD treebank, which forms the basis for a portion of PDBUD. It contains approximately 8,000 trees.
  • PDB (Polish Dependency Bank): A new version of the Polish Dependency Bank, from which 14,000 trees were automatically converted for PDBUD.

Key Insights and Ideas

  • Size Matters for Rich Languages: The paper strongly suggests that for morphologically rich languages like Polish, a larger dataset (like PDBUD) is essential for achieving high parsing performance, even with advanced graph-based parsers.
  • UD Schema Flexibility: The need to extend the UD schema with new relation subtypes highlights the ongoing challenge of creating a truly universal annotation scheme that can accommodate the diverse linguistic phenomena across languages.
  • Enhanced Graphs for Coordination: The introduction of enhanced edges for coordinated structures is a significant step towards capturing more complex syntactic relationships that are often simplified or lost in standard dependency representations.
  • Graph-Based Parsers Excel: The evaluation results consistently show that graph-based dependency parsers outperform transition-based parsers for Polish, indicating that their ability to consider global sentence structure is beneficial for this language.

Key Results

  • Largest Polish UD Treebank: PDBUD is the largest Polish dependency bank in UD format, with 22,208 trees and 351,715 tokens.
  • Improved Parsing Performance: Experiments showed that PDBUD is large enough to train high-quality graph-based dependency parsers for Polish.
  • Graph-Based Parsers Outperform: In evaluation, graph-based parsers like the Stanford parser achieved the best LAS (Labeled Attachment Score) of 90.03 on gold-standard POS tags, and 88.04 LAS on predicted POS tags. This is slightly lower than previous state-of-the-art results, attributed to the inclusion of longer sentences in the PDBUD test set.
  • Morphosyntactic Prediction: The Stanford system also demonstrated strong performance in predicting UPOS (Universal Part-of-Speech tags) and XPOS (Polish-specific tags), achieving 97.87% and 92.45% F1 scores, respectively.

Limitations and Future Work

  • Annotation Decisions: The authors acknowledge that some of their annotation decisions might be debatable and could benefit from further discussion within the UD community.
  • Elliptical Constructions: The paper notes that many elliptical constructions in Polish are currently labeled as "orphan" in PDBUD.
  • Future Work:
    • Plan to add empty nodes to represent elided elements in PDBUD trees to better handle ellipsis.
  • Aim to create a Polish version of the Parallel Universal Dependency treebank.
  • PDBUD data has already been used in shared tasks, indicating its immediate utility.
  • The authors intend to replace the older PL-SZ treebank with the corrected, extended, and enhanced PDBUD.

Summary updated

BibTeX
@inproceedings{wroblewska2018extended,
  author = {Alina Wróblewska},
  title = {{Extended and Enhanced Polish Dependency Bank in Universal Dependencies Format}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
}
Figure