Synonymy in Bilingual Context: The CzEngClass Lexicon

Zdeňka Urešová, Eva Fučíková, Eva Hajičová, Jan HajičCOLING 2018

TL;DR

This paper introduces CzEngClass, a novel bilingual lexicon that captures verb synonymy between Czech and English. It addresses the challenge of defining and identifying synonyms in a cross-lingual context by leveraging parallel corpora and existing lexical resources, aiming to provide a gold standard for future NLP research and tools.

What Problem Does This Paper Address?

Identifying and representing verb synonymy, especially across languages, is a complex task in Natural Language Processing (NLP). While resources like WordNet and FrameNet exist for individual languages, they often lack the depth needed for bilingual synonymy and don't always link semantic roles to verb arguments effectively. This paper tackles the gap of creating a structured, bilingual lexicon that explicitly defines verb synonymy based on semantic roles and actual usage in parallel texts, bridging the divide between monolingual resources and the practical needs of cross-lingual NLP.

Methods

The CzEngClass lexicon is built using a semi-automatic process. The core methodology involves:

  • Seed Verb Selection: Identifying a set of Czech verbs from the Prague Czech-English Dependency Treebank (PCEDT) representing various frequencies.
  • Automatic Alignment: Using the PCEDT and the CzEngVallex lexicon to automatically find potential English translations for these Czech "seed" verbs.
  • Manual Pruning and Class Formation: Human annotators then manually prune these potential translations to form bilingual synonym classes. This involves comparing valency frames and semantic roles.
  • Semantic Role (SR) Mapping: A crucial step is defining a common set of semantic roles for each synonym class. Each verb member's arguments are then mapped to these SRs, ensuring a consistent semantic representation across synonyms.
  • Linking to External Resources: Entries are linked to existing resources like FrameNet, VerbNet, WordNet, and PDT-Vallex to enrich their semantic and syntactic information.

Datasets

  • Prague Czech-English Dependency Treebank (PCEDT 2.0): This is the primary resource, a parallel corpus with over 1.2 million words and 50,000 sentences per language, annotated with deep syntactico-semantic dependency trees. It's used for identifying verb translations and providing corpus examples.
  • CzEngVallex: A bilingual Czech-English valency lexicon derived from the PCEDT, used for aligning verb senses and arguments.
  • PDT-Vallex and EngVallex: Monolingual valency lexicons for Czech and English, respectively, used for valency information and word sense labeling.
  • External Lexical Resources:
    • English: FrameNet, PropBank, VerbNet, OntoNotes, English WordNet.
  • Czech: Czech WordNet, VALLEX.

Key Insights and Ideas

  • Context is King for Synonymy: The paper strongly emphasizes that synonymy, especially in a bilingual context, is not absolute but highly dependent on the specific semantic roles (SRs) a verb expresses with its arguments in a given context.
  • Bridging Monolingual Gaps: CzEngClass aims to bridge the gap left by monolingual resources by explicitly defining bilingual synonymy and linking semantic roles across languages, which is crucial for cross-lingual understanding.
  • Semi-Automatic Approach is Necessary: While automation helps in initial candidate generation, manual annotation and adjudication are indispensable for achieving high-quality synonym classes, particularly for filtering and role assignment.
  • Interannotator Agreement Challenges: The paper highlights that traditional IAA metrics can be misleading in biased tasks (like synonym selection where there's a strong preference to keep candidates). New ways of measuring consensus and deviation are explored.

Key Results

  • The first version of CzEngClass contains 200 classes, covering approximately 1800 verbs.
  • Interannotator agreement (IAA) was evaluated using Cohen's kappa and Fleiss' kappa. The results showed lower agreement than expected, attributed partly to a high bias (around 5:1) favoring the inclusion of candidate verbs.
  • Despite low kappa scores, consensus analysis revealed that a majority agreement was reached for nearly 90% of datapoints, and full agreement among annotators for about 66% of cases, indicating that adjudication effort, while necessary, is manageable.
  • Deviation analysis showed that annotators exhibited different levels of "conservatism" or "liberalism" in class membership decisions.

Limitations and Future Work

  • Bias in IAA: The paper acknowledges that the IAA metrics are affected by the inherent bias in the task, where annotators are more inclined to keep candidate verbs than to discard them.
  • Mapping Complexity: The paper notes that the mapping of valency slots to semantic roles might be m:n (many-to-many) rather than 1:1, which influences interannotator agreement and requires careful definition.
  • Future Work:
    • Completing Phase 2 annotation, which involves assigning semantic roles to classes and detailed mapping of valency to these roles.
  • Extending CzEngClass to all verbs in the PCEDT.
  • Using CzEngClass as a gold standard for experiments on automatic synonym class extraction from larger datasets.
  • Extending the approach to other languages.

Summary updated

BibTeX
@inproceedings{uresova2018synonymy,
  author = {Zdeňka Urešová and Eva Fučíková and Eva Hajičová and Jan Hajič},
  title = {{Synonymy in Bilingual Context: The CzEngClass Lexicon}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2018},
}
Figure