TL;DR
This paper explores how the "subjectivity" of a word's meaning (whether it carries opinion or is purely factual) impacts its translation in a cross-lingual context. They found that considering this subjectivity, even with a simplified "subjective/objective" classification, significantly improves the accuracy of translating words in context, sometimes as well as using full, fine-grained word sense disambiguation.
What Problem Does This Paper Address?
Translating words accurately when they appear in different contexts is a long-standing challenge in Natural Language Processing (NLP). This is especially true for "cross-lingual lexical substitution," where the goal is to find the correct translation of a word in a target language (like Chinese) given its context in a source language (like English). While traditional approaches focus on identifying the precise meaning (sense) of a word, this paper hypothesizes that the sentiment or subjectivity of that sense plays a crucial role. For instance, "positive" can mean "greater than zero" (objective) or "advantageous/good" (subjective). The authors argue that these different flavors of meaning, particularly the subjective ones, need to be preserved in translation to maintain the original intent and nuance. They aim to demonstrate that explicitly modeling this subjectivity can enhance cross-lingual translation performance, especially for words that have both subjective and objective meanings.
Methods
The core of this work involves building and evaluating different systems for English-Chinese lexical substitution. They start with a Baseline system that simply assigns the most frequent translation. Then, they introduce System B, which uses standard contextual features like surrounding words (lemmatized bag-of-words), part-of-speech tags, collocations, and syntactic relations derived from dependency parsing.
The novel contribution lies in how they incorporate subjectivity features. They explore two main ways:
- B+gold subj: This uses human-annotated "gold standard" labels indicating whether a word's sense is subjective or objective. This serves as an upper bound for their approach.
- B+auto subj: This uses an automatically predicted subjectivity label derived from a separate "Subjectivity Word Sense Disambiguation" (SWSD) model. SWSD is a binary classifier that determines if a word is used in a subjective or objective sense.
For comparison, they also incorporate full fine-grained word sense information (both gold-standard and automatically predicted) in B+gold senses and B+auto senses. All systems are trained using Support Vector Machines (SVMs) with a linear kernel.
Datasets
The authors construct their English-Chinese lexical substitution gold standard by translating the English target words from the SENSEVAL 2 and SENSEVAL 3 lexical sample training and test sets. These datasets are chosen for their domain independence. The translation process involved two native Chinese speakers with good English command, using the online dictionary iciba. Candidate translations were provided, and annotators assigned the most fitting ones. To focus on the impact of subjectivity, they specifically selected 28 subjectivity-ambiguous words (words with both subjective and objective senses). This resulted in 2890 training sentence pairs and 1444 test sentence pairs. The subjectivity of the SENSEVAL senses themselves was annotated by a separate near-native English speaker.
Key Insights and Ideas
- Subjectivity is a powerful signal for translation: The central idea is that preserving the subjective or objective nature of a word's meaning is crucial for accurate cross-lingual translation, especially for words that can be used in both ways.
- Coarse-grained subjectivity is often sufficient: The experiments suggest that a simple binary classification of "subjective" vs. "objective" can be highly effective, sometimes performing as well as or even better than using full, fine-grained word sense disambiguation. This implies that for certain tasks, over-disambiguating might introduce noise.
- SWSD can boost performance: Automatically predicting subjectivity using SWSD significantly improves lexical substitution accuracy compared to a basic system, demonstrating the practical utility of this approach.
- Subjectivity-ambiguity is common: The paper highlights that words with both subjective and objective senses are frequent (around 30% of common words), making this a relevant problem for a substantial portion of the lexicon.
Key Results
The paper reports significant improvements in lexical substitution accuracy:
- The standard system B (with contextual features) improved accuracy by nearly 11% on average compared to the most frequent translation baseline.
- Incorporating gold-standard subjectivity (B+gold subj) led to a further strong improvement, confirming the hypothesis.
- B+auto subj (using automatically predicted subjectivity) performed significantly better than B, showing a substantial gain.
- Interestingly, B+auto subj performed better than B+auto sense (using automatically predicted fine-grained senses), suggesting that the coarse-grained subjectivity information is more robust or easier to leverage for this task than complex sense disambiguation.
- The difference between potential performance (B+gold subj) and actual performance (B+auto subj) is attributed to the imperfect accuracy of the SWSD component, likely due to data biases.
Limitations and Future Work
Limitations: * For some words (e.g., "authority," "stress"), the subjectivity feature did not improve performance. This is attributed to either a single Chinese translation covering both objective and subjective senses, or the need for finer-grained disambiguation than just subjective/objective. * The automatic SWSD component, while effective, is not perfect, leading to a gap between potential and actual performance gains. This is partly due to distributional biases in the training data.
Future Work: * Expand the study to a wider range of words and languages. * Improve the accuracy of automatic SWSD models. * Investigate the integration of unsupervised subjectivity features. * Explore the feasibility of using existing parallel texts instead of dedicated human-translated data for training.
Summary updated