TL;DR
This paper introduces PAWS, a novel multi-lingual parallel treebank with detailed coreference annotations across English, Czech, Russian, and Polish. PAWS aims to facilitate cross-lingual research on how languages express coreference, offering a rich resource for linguistic analysis and the development of computational models.
What Problem Does This Paper Address?
Understanding how different languages express coreference – the relationship between linguistic expressions that refer to the same entity – is crucial for both theoretical linguistics and computational natural language processing (NLP). While some multilingual corpora with coreference exist, they often lack depth or broad language coverage. Specifically, existing resources might focus on only a few languages, lack detailed syntactic information, or not account for phenomena like zero anaphora (where subjects or objects are omitted but understood). This gap hinders comprehensive cross-lingual studies and the development of robust multilingual coreference resolution systems. PAWS aims to bridge this gap by providing a richly annotated parallel corpus across four diverse languages, including Slavic languages known for their pro-drop capabilities.
Methods
The PAWS treebank is built upon the Prague Dependency Treebank (PDT) 2.0, extending its tradition of multilingual coreference annotation. The coreference annotation is performed manually on the tectogrammatical layer, which represents deep syntactic structure. This layer allows for the annotation of "zero anaphora," crucial for languages like Czech and Polish where subjects can be omitted.
Key methodological aspects include:
- Tectogrammatical Layer: This deep syntactic representation captures overt and elided content words, enabling the annotation of coreferential relations even for dropped pronouns.
- Manual Annotation: Coreference relations are annotated by human experts following the Prague coreference annotation style.
- Cross-lingual Alignment: Word alignment links are provided between all language pairs on both surface and tectogrammatical layers, facilitating cross-lingual analysis. These alignments are largely automatic but complemented by manual annotation for specific coreferential expressions.
- Annotation Schema: The schema covers grammatical coreference (e.g., relative and reflexive pronouns) and textual coreference (e.g., pronominal coreference, textual ellipsis, nominal coreference, and anaphoric adverbs).
Datasets
The primary dataset introduced is the PAWS treebank. It consists of:
- English texts: Extracted from the Wall Street Journal section of the Penn Treebank, specifically from the Prague Czech-English Dependency Treebank 2.0 (PCEDT 2.0).
- Czech, Russian, and Polish texts: These are translations of the English Wall Street Journal texts. The Russian and Polish texts were translated by native speakers and subsequently revised.
PAWS builds upon existing resources like the PCEDT 2.0 and its coreferential extension. It is distinct from corpora like ParCor 1.0, which focuses on German-English and annotates only pronominal coreference, whereas PAWS annotates full coreference chains.
Key Insights and Ideas
- The Importance of Zero Anaphora: The inclusion of the tectogrammatical layer is vital for accurately annotating coreference in pro-drop languages like Czech and Polish, where omitted subjects and objects are common.
- Cross-lingual Differences in Coreference Expression: The paper highlights that even closely related Slavic languages exhibit significant differences in coreference phenomena, such as the degree of pro-drop usage and the variety of reflexive pronoun usage. PAWS provides the data to study these differences systematically.
- Annotation Schema Applicability: The project tests the applicability of a single annotation schema across multiple languages, contributing to the broader effort of developing universal linguistic representations.
- Potential for NLP Research: PAWS is not only a linguistic resource but also a valuable benchmark for developing and evaluating cross-lingual automatic coreference resolution systems.
Key Results
The paper presents statistics on the PAWS treebank, offering insights into language-specific differences:
- Tectogrammatical Nodes: Czech has a larger number of tectogrammatical nodes compared to the other languages, potentially due to a tendency to use finite subordinate clauses more frequently.
- Coreferring Nodes: Czech also shows the highest number of coreferring nodes, correlating with its richer tectogrammatical annotation and frequent use of personal constructions.
- Grammatical Coreference: Russian has the largest number of grammatical coreference relations, possibly due to its extensive use of infinitive constructions where subjects are controlled by governing verbs.
- Textual Pronominal Coreference: English exhibits the most overt textual pronominal coreference, reflecting its non-pro-drop nature, while Czech is highly pro-drop, making anaphoric pronoun use less typical. Polish and Russian show intermediate pro-drop qualities.
- Nominal Coreference: Russian appears to realize coreference more frequently through nominal groups, a finding that warrants further analysis and may be a translation effect.
Limitations and Future Work
- Polish Annotation: While English, Czech, and Russian pronouns and zeros have manual cross-lingual alignment, Polish is not covered by manual alignment yet.
- Simplification in Russian/Polish Annotation: The tectogrammatical annotation for Russian and Polish is noted as being more simplified than for English and Czech, particularly regarding obligatory valency positions and identification structures.
- Ambiguity in Near-Identity: Coreference relations, especially in cases of "near-identity," can be ambiguous, and annotator decisions can be influenced by explicit anaphoric markers.
- Future Work: The authors suggest that PAWS can be further leveraged for cross-lingual studies and for developing and evaluating multilingual NLP systems. Future work could involve extending the Polish annotation and conducting deeper comparative analyses.
Summary updated