Bilingual lexicon extraction for a distant language pair using a small parallel corpus

Ximena Gutierrez-VasquesNAACL 2015

TL;DR

This paper proposes a novel method for extracting bilingual lexicons between distant language pairs with limited parallel data. It tackles the challenge of low-resource languages by leveraging morpheme-based representations and a unique approach to contextual information that bypasses the need for a pre-existing seed lexicon.

What Problem Does This Paper Address?

Building bilingual lexicons is crucial for many natural language processing (NLP) tasks, such as machine translation. However, traditional methods often rely on large parallel corpora and robust word alignment techniques, which are not feasible for many language pairs, especially those that are typologically distant and lack extensive digital resources. This paper focuses on the Spanish-Nahuatl language pair, a prime example of a low-resource setting where obtaining sufficient parallel or even monolingual data is difficult. The authors highlight that existing dictionaries may be outdated or lack standardization due to dialectal variations, making them unreliable for NLP applications. The core problem is how to effectively extract bilingual lexical information when faced with scarce data and significant linguistic differences, without relying on pre-compiled, language-specific resources.

Methods

The proposed method aims to overcome the limitations of traditional approaches by:

  • Morpheme-Based Representations: Instead of using whole words, the authors propose to break down words into their constituent morphemes. This is particularly beneficial for morphologically rich languages like Nahuatl, where a single word can convey complex meanings. Morpheme-based representations help reduce data sparseness and facilitate the identification of lexical correspondences.
  • Contextual Information without Seed Lexicons: A key innovation is the method for utilizing contextual information without requiring a pre-existing bilingual seed dictionary. The authors propose to analyze the distribution and dispersion of morpheme positions within a text. The hypothesis is that translations tend to appear in similar positions and exhibit similar distribution patterns. This positional information is used to compare contextual vectors and identify potential translation candidates.
  • Combining Features: The approach combines contextual features with association measures. This multi-faceted strategy aims to improve the robustness and accuracy of lexicon extraction, especially in noisy or limited data scenarios. The authors plan to develop a weighting scheme to combine these different cues for ranking translation candidates.

Datasets

The paper introduces a newly constructed Spanish-Nahuatl parallel corpus. This corpus was built from various sources, including non-digital books, and required significant manual correction and OCR processing due to the challenges of digitizing Nahuatl texts. The authors also mention the use of normalization rules for classical Nahuatl to mitigate dialectal and orthographical variations. For evaluation, they suggest using existing methods suitable for parallel corpora and potentially testing on other resource-rich distant language pairs like Spanish-German to assess competitiveness.

Key Insights and Ideas

  • Morphemes as a Bridge: The paper posits that focusing on morphemes, rather than whole words, can be a powerful strategy for bridging typologically distant languages, especially when dealing with morphologically complex ones.
  • Positional Cues for Lexicon Induction: The idea of using the distribution and dispersion of morpheme positions as a proxy for semantic similarity is a novel approach to bypass the need for seed lexicons. This is a significant departure from methods that rely on direct word-to-word mapping.
  • Low-Resource Focus: The entire methodology is geared towards low-resource settings, acknowledging that many language pairs face similar data scarcity issues. This makes the proposed approach broadly applicable beyond the specific Spanish-Nahuatl case.
  • Circumventing the "Chicken-and-Egg" Problem: The authors directly address the common challenge where bilingual lexicon extraction methods require a seed lexicon, but creating a seed lexicon often requires bilingual data. Their positional analysis aims to break this cycle.

Key Results

Not discussed in the paper. The paper is a thesis proposal, outlining the planned methodology and research direction.

Limitations and Future Work

  • Data Scarcity: The primary limitation acknowledged is the small size of the Spanish-Nahuatl parallel corpus (around 1 million tokens), which is still considered very small for traditional NLP approaches.
  • Orthographic Variation: Despite normalization efforts, dialectal and orthographical variations within Nahuatl remain a challenge.
  • Synonyms and Polysemy: The authors note that dealing with synonyms and polysemic words is an aspect that needs to be tackled in future work.
  • Future Work: The paper suggests exploring different experimental setups for contextual vector representations, including various association measures, weighting schemes, and context window sizes. They also propose evaluating the method on other language pairs, including resource-rich distant pairs, to assess its generalizability and competitiveness. The paper also mentions the need to decide whether to use individual morpheme representations or composed representations of words based on morphemes.

Summary updated

BibTeX
@inproceedings{gutierrezvasques2015bilingual,
  author = {Ximena Gutierrez-Vasques},
  title = {{Bilingual lexicon extraction for a distant language pair using a small parallel corpus}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2015},
}
Figure