TL;DR
This paper introduces SUD, a new surface-syntactic annotation scheme designed to be "near-isomorphic" to Universal Dependencies (UD). SUD aims to provide a more linguistically grounded annotation by adhering to distributional criteria for syntactic structure and relation labeling, offering a valuable alternative for typological research and a potential tool for improving UD treebank quality.
What Problem Does This Paper Address?
Universal Dependencies (UD) is a highly successful project aiming to create a consistent, cross-linguistic annotation scheme for dependency parsing. However, UD's design choices, such as favoring content words as heads and simplifying structures, have been criticized for potentially making treebanks less informative for typological research and for flattening linguistic nuances. Specifically, UD's approach can obscure the role of function words and lead to less intuitive representations of coordination. This paper addresses the need for an annotation scheme that retains the benefits of UD's cross-linguistic applicability while also being more faithful to surface-syntactic principles and distributional evidence, thereby offering a richer resource for linguistic analysis.
Methods
The core contribution is the SUD (Surface-Syntactic Universal Dependencies) annotation scheme. SUD is built upon surface-syntactic criteria derived from linguistic theories like Meaning-Text Theory and Word Grammar. Key methodological shifts from UD include:
- Function Words as Heads: Unlike UD, SUD designates function words (e.g., prepositions, auxiliaries, copulas) as heads in many cases, reflecting their syntactic role in structuring phrases.
- Distributional Criteria for Relations: Dependency relations are defined based on the distribution of words and their ability to form independent units (autonomizability), rather than solely on the category of the dependent.
- Simplified Relation Labels: SUD aims to reduce redundancy by grouping similar UD relations under more general labels (e.g.,
subjfor bothnsubjandcsubj). - Chaining Analysis for Coordination: SUD adopts a "chaining" approach to coordination, attaching conjuncts to their closest neighbors, which is argued to better reflect the dependency length minimization principle, contrasting with UD's "bouquet" analysis.
- Bi-directional Conversion: The authors developed rule-based graph transformation grammars using the Grew platform to enable seamless conversion between UD and SUD. This conversion process is also proposed as a valuable error-mining tool for UD treebanks.
Datasets
The paper evaluates the conversion between UD and SUD using the UD 2.2 treebanks. The evaluation was performed on the test portion of each of the 122 corpora available in UD 2.2. No new benchmark datasets were introduced; instead, the focus was on the conversion quality of existing UD treebanks.
Key Insights and Ideas
- Surface Syntax Offers a Complementary View: SUD demonstrates that a surface-syntactic perspective, prioritizing distributional evidence and function-word headedness, can offer a linguistically richer and less redundant annotation than UD, particularly for phenomena like coordination and the role of function words.
- Isomorphism as a Quality Control Tool: The development of bi-directional conversion grammars between UD and SUD serves as a powerful mechanism for identifying inconsistencies and potential errors within UD treebanks. Treebanks that are unstable under this double conversion often highlight non-standard UD annotation practices.
- Dependency Length Minimization: The paper argues that SUD's chaining analysis for coordination aligns better with the principle of minimizing dependency length, which is hypothesized to reduce cognitive load during language processing.
- Potential for Typological Research: By adhering more closely to traditional syntactic analyses and preserving more nuanced distinctions, SUD is presented as a more suitable resource for cross-linguistic typological studies of syntax.
Key Results
The evaluation of the UD → SUD → UD conversion on the UD 2.2 treebanks showed high scores, indicating good convertibility:
- Median LAS (Labeled Attachment Score): 94.1%
- Median UAS (Unlabeled Attachment Score): 98.8%
However, the paper also highlights that discrepancies arise, particularly due to inconsistencies between relation names and the Part-of-Speech (POS) of their dependents in UD. For example, UD's amod relation often converts to SUD's dep when the dependent is not an adjective, and the reverse conversion struggles to recover amod in such cases. The lowest LAS scores were observed for UD_Korean-Kaist (71.8%), UD_Japanese-BCCWJ (74.0%), and UD_Japanese-GSD (74.4%). The authors note that incorporating features like MWEPOS can improve conversion scores.
Limitations and Future Work
- Near-Isomorphism, Not Perfect: The authors acknowledge that SUD is "near-isomorphic" to UD, not perfectly so, due to fundamental differences in their underlying principles.
- Language-Specific Rules: While a universal core rule set for conversion exists, language-specific rules are necessary for optimal performance, and only French has been fully implemented.
- Inconsistencies in UD: The conversion process revealed significant inconsistencies within UD treebanks, particularly regarding the relationship between relation labels and dependent POS tags. This suggests a need for refining UD annotation guidelines or correcting existing treebanks.
- Future Work: The authors plan to:
- Make SUD treebanks accessible.
- Explore further improvements to conversion rules, potentially by introducing new features like UDPOS to handle unexpected POS assignments.
- Develop new treebanks directly in SUD and then convert them to UD, using the conversion as a quality control mechanism.
- Investigate the impact of these inconsistencies on parser performance and explore how SUD can contribute to the upcoming UD 3.0 annotation scheme.
Summary updated