TL;DR
This paper introduces a novel unsupervised method for translating multiword expressions (MWEs) by treating translation as a process of contextualizing word meanings within a bilingual vector space. Their "co-compositionality" approach, which considers how words mutually influence each other's meaning in a dependency, significantly outperforms existing methods and a neural machine translation baseline.
What Problem Does This Paper Address?
Translating multiword expressions (MWEs) – phrases where the meaning isn't simply the sum of its parts, like "hard drug" or "take a cab" – is a significant challenge in natural language processing. Traditional machine translation systems often struggle with these idiomatic or collocated phrases. While compositional semantics aims to model how word meanings combine, applying this to unsupervised cross-lingual translation, especially for MWEs, remains an open problem. Existing approaches often treat word combinations in a one-way fashion, failing to capture the mutual influence between words in a phrase.
Methods
The core of the proposed method is a syntax-sensitive compositional distributional strategy. Instead of representing a phrase as a single vector, it models the meaning of each constituent word as a contextualized vector. This contextualization is achieved by considering the lexico-syntactic contexts and selectional preferences of words within a dependency relationship.
Specifically, for a dependency between a head word (h) and a dependent word (d) (e.g., adjective-noun), the method computes two contextualized senses: one for the head influenced by the dependent, and one for the dependent influenced by the head. This is done by adding the word's original vector to a vector representing the selectional preferences it imposes or receives from its syntactic partner. These selectional preferences are derived from the most salient contexts of the word.
For translation, this approach is extended to a bilingual setting. First, candidate translations are generated using cross-lingual word embeddings. Then, the compositional meaning of the source dependency and each target candidate is computed using the contextualized vector approach. Finally, the translation candidate whose compositional meaning is most similar to the source dependency's meaning (measured by cosine similarity) is selected. This "co-compositionality" approach, where both words in a dependency mutually contextualize each other, is the key innovation.
Datasets
The paper introduces a new freely available dataset of 273 English-Spanish compound equivalents. This dataset comprises MWEs with varying degrees of semantic compositionality, specifically focusing on adjective-noun (amod) and verb-noun (vobj) dependencies. It was generated by leveraging parallel corpora and bilingual word embeddings, then manually reviewed and enlarged. The dataset includes examples ranging from free combinations to collocations and idioms.
Key Insights and Ideas
- Co-compositionality is Crucial for MWE Translation: The paper strongly advocates for and demonstrates that considering the mutual influence between words in a dependency (co-compositionality) is significantly more effective for translating MWEs than one-way contextualization.
- Translation as Contextualization: Framing translation as a process of contextualizing word meanings within a bilingual vector space offers a powerful unsupervised approach.
- Syntax-Sensitive Meaning Representation: Leveraging syntactic dependencies and selectional preferences provides a more nuanced representation of word meaning in context compared to purely distributional methods.
- Bilingual Embeddings as a Bridge: Cross-lingual word embeddings are essential for bridging the gap between source and target languages, enabling the generation of candidate translations and the comparison of contextualized meanings.
Key Results
The proposed CT(head+dep) method achieved significantly higher accuracy on the English-Spanish MWE translation task compared to several baselines and a state-of-the-art neural machine translation system (UNdreaMT).
- Accuracy: CT(head+dep) achieved an accuracy of 0.847 for amod expressions and 0.843 for vobj expressions.
- Comparison to Baselines: This significantly outperformed UNdreaMT (0.543/0.571), CT(dep) (0.510/0.564), CT(head) (0.462/0.400), add (0.543/0.564), mult (0.354/0.505), and corpus (0.326/0.297).
- Effectiveness of Co-compositionality: The results clearly show that combining both head and dependent contextualizations (CT(head+dep)) is superior to using only one (CT(head) or CT(dep)).
Limitations and Future Work
The authors acknowledge several limitations and propose future directions:
- Fertile Translations: The current method struggles with translations where the target term has a different number of words or syntactic structure than the source. Future work will involve expanding candidate extraction strategies to handle these cases.
- Distinguishing Compositionality: The method does not currently distinguish between compositional and non-compositional expressions. Future work aims to first identify the degree of compositionality of an MWE before applying the appropriate translation strategy.
- Complex Expressions: The current approach is limited to binary dependencies. The ultimate goal is to generalize the method to handle arbitrary sentence structures, leading to an unsupervised machine translation system.
- Error Analysis: The paper identifies error types such as "DistSimil" (semantic similarity issues), "Conventions" (non-collocating expressions), "Translation" (approximate translations), and "Idiomacity." Addressing these will be crucial for further improvement.
Summary updated