Abstract: Chenglei Si, Kui Wu, Ai Ti Aw, Min-Yen Kan. Proceedings of the 6th Workshop on Asian Translation.
2019.
Introduction
Standard Neural Machine Translation (NMT) systems typically strive for a single "correct" translation. However, natural language is replete with "sentiment ambiguous lexicons"—words whose meaning and polarity shift dramatically based on context. For instance, the English word proud can translate to the Chinese 自豪 (zìháo, feeling honored) in a positive sense, or 骄傲 (jiāo'ào, being arrogant) in a negative one.
Current NMT models often struggle with these nuances because they are trained to produce the most statistically probable output. If a training set is biased toward one sentiment, the model will default to it, even when the intended meaning is different. This is particularly problematic for short-form user-generated content, such as reviews or social media posts, where preserving the original "vibe" is essential.
Methodology: Integrating Sentiment Awareness
The authors propose a two-stage pipeline to address this. First, a BERT-based sentiment classifier annotates source sentences with sentiment labels. Second, these labels are integrated into a Sequence-to-Sequence (Seq2Seq) NMT model using three strategies:
- AddLabel: Prepending a sentiment token (e.g.,
<pos>or<neg>) to the beginning of the source sentence. - InsertLabel: Inserting the sentiment token directly before the ambiguous word.
- Valence-Sensitive Embedding (VSE): The most sophisticated approach, where the model maintains two distinct embedding vectors for an ambiguous word—one for its positive sense and one for its negative sense. The model selects the appropriate vector during training and inference based on the sentiment label.
While AddLabel and InsertLabel act as external hints, VSE fundamentally alters the model's internal representation of a word's meaning by providing dedicated vector space for different polarities.
Experimental Setup
The pipeline was evaluated using several benchmarks:
- Sentiment Classification: BERT was fine-tuned on standard benchmarks including SST-2, Yelp, and IMDB.
- Translation Training: 210K English-Chinese pairs containing 110 pre-identified ambiguous words were extracted from MultiUN, TED, and AI Challenger corpora.
- Ambiguous Test Set: To test the models' reliance on sentiment labels, the authors created a set of 120 short sentences lacking contextual clues (e.g., "She is so stubborn"), forcing the model to rely on the provided label.
Key Insights and Results
The VSE method consistently outperformed labeling strategies, suggesting that providing distinct "slots" in memory for different polarities is more effective than steering a single representation with prefix tokens.
Visual analysis using t-SNE confirmed that VSE successfully learns to separate meanings in vector space. For example, the negative embedding for "stubborn" clusters near "obstinate," while the positive version moves toward "tenacious."
Key performance metrics include:
- Sentiment Matching: VSE achieved 31.5% accuracy for positive and 69.4% for negative translations on the ambiguous test set, far exceeding the baseline (18.0% and 50.4%).
- Translation Quality: VSE reached the highest BLEU scores (42.38 positive, 56.88 negative) on the test set.
- Precision: VSE achieved a word-level F1 score of 34.2 in contextual tests, compared to the baseline's 33.7.
Conclusion
This research demonstrates that external sentiment knowledge can effectively disambiguate translations in low-context scenarios. While currently limited by classifier accuracy and a fixed lexicon of 110 words, the VSE approach provides a robust framework for sentiment-aware NMT. Future work may involve scaling this to broader lexicons and integrating VSE into Transformer-based architectures.
Summary updated