TL;DR
This paper introduces "Cognate Morfessor," a novel method for segmenting words into morphemes in multilingual neural machine translation. By explicitly modeling cognates (words with shared origins across languages), it improves translation quality, especially for low-resource languages, by leveraging cross-lingual consistency in word segmentation.
What Problem Does This Paper Address?
Modern neural machine translation systems often rely on subword units (like Byte Pair Encoding or BPE) to handle rare words and morphological variations. However, these methods can struggle with consistency, especially when translating between related languages with shared vocabulary (cognates). When cognates are segmented differently in the source and target languages, the translation system misses opportunities to leverage this shared linguistic heritage. This inconsistency can lead to suboptimal translations, particularly for languages with fewer training data resources, where the system has less opportunity to learn robust representations. The paper aims to bridge this gap by creating a segmentation system that is "cognate-aware," ensuring that similar words across languages are segmented in a harmonized way.
Methods
The core contribution is Cognate Morfessor, a multilingual variant of the Morfessor method. Unlike standard Morfessor, which operates on monolingual data, Cognate Morfessor is trained on a bilingual corpus to segment words in two target languages simultaneously. It extends the Morfessor cost function to include an "edit cost" component that explicitly models the differences (Levenshtein edits) between cognate morphs. This encourages the model to learn regular patterns of sound and spelling changes between cognates.
Key aspects of the method include:
- Cognate Extraction: Cognates are automatically extracted from parallel data using word alignment and Levenshtein distance filtering.
- Multilingual Training: The model learns segmentations for two target languages concurrently, allowing for cross-lingual consistency.
- Edit Modeling: A Levenshtein-based edit cost is incorporated to penalize inconsistent segmentations of cognates and to learn regular sound/spelling transformations.
- Language-Agnostic: The method does not require language-specific annotations like lemmatizers or parsers.
Datasets
The authors used the WMT18 News Translation Shared Task data. Specifically, they created a Finnish-Estonian parallel dataset by triangulating from English-Finnish and English-Estonian parallel data. This resulted in approximately 679,252 sentence pairs. For training the NMT system, they utilized 9 million multilingual sentence pairs, comprising English-Finnish and English-Estonian data, including both parallel and synthetic backtranslated data.
Key Insights and Ideas
- Cognates are a valuable linguistic resource: Explicitly modeling cognates can significantly improve multilingual translation by leveraging shared linguistic structures.
- Segmentation consistency matters: Harmonizing subword segmentation for cognates across languages leads to better translation quality, especially for low-resource languages.
- Levenshtein distance for cognate edits: Using edit distance to model the transformations between cognate morphs is an effective way to capture regular sound/spelling changes.
- Multilingual training benefits even high-resource languages: The experiments showed that even for Finnish, which has more data, multilingual training with Cognate Morfessor yielded improvements.
Key Results
The paper reports that their main system, using Cognate Morfessor segmentation, outperforms baseline systems (multilingual BPE and monolingual Morfessor) on both Estonian and Finnish for character-F scores. For Estonian, the benefit was particularly pronounced, highlighting the effectiveness for low-resource languages. For Finnish, results were competitive, with improvements seen in chrF-1.0 scores. They also experimented with ensemble decoding and found that combining multiple models further boosted performance.
Limitations and Future Work
The authors acknowledge that Cognate Morfessor, in its current form, is limited to at most two target languages due to the model structure. They also note that their cognate extraction process used unweighted Levenshtein distance, and suggest that using weighted or graphonological Levenshtein distance could be a future improvement for more nuanced cognate modeling. Future work could also explore combining various configurations (ensemble, finetuning, LM filtering, etc.) more extensively.
Summary updated