Abstract: In Machine Translation (MT), using compound words or phrases often makes the translation process easier. For example, the phrase "高速公路" corresponds unambiguously to “information highway”. It is not necessary to break it down to "信息" (information), "高速" (highspeed) and "路" (road). However, some compound words (phrases) in Chinese are composed of simpler words which can play significantly different roles in sentences when they are broken down. For example, the compound word "机器翻译" (machine translation) may be broken into "机器" (machine) and "翻译" (translate), as in the sentence "他用机器翻译论文" (He uses a machine to translate papers). We call such a compound word “Sensitive Word”. During Chinese MT processing, if the first segmentation result in which a sensitive word is segmented as a single word leads to a failure, the alternative solution with the sensitive word broken down is considered as the preferred one. This allows us to reach at a higher efficiency by avoiding examining unlikely segmentation solutions. In this paper, we describe the problems related to sensitive words. A machine readable dictionary has been examined, and 764 sensitive words have been found among 87,600 words. This shows that sensitive word is a common phenomenon in Chinese that is worth closer examination.
TL;DR
The paper identifies "Sensitive Words" in Chinese—compound words that, when segmented differently, radically change a sentence's syntactic structure. By analyzing a dictionary of 87,600 entries, the authors found that nearly 1% of words are "sensitive," providing a targeted strategy to improve Machine Translation (MT) efficiency by knowing exactly which words to re-examine when a translation fails.
What Problem Does This Paper Address?
In languages like English, spaces clearly demarcate words. In Chinese, sentences are continuous strings of characters. The first step in any Chinese NLP task is segmentation—deciding where one word ends and the next begins.
At the time of writing, many researchers claimed Chinese segmentation was a "solved problem" with 99% accuracy. Ren and Nie argue this is a myth. High accuracy in isolation doesn't mean the segmentation is useful for downstream tasks like Machine Translation.
The core challenge is combinatory ambiguity. For example, the string "将来" (jiānglái) usually means "future" (a noun/adverb). However, in the sentence "He will come to study," those same characters function as "will" (jiāng) and "come" (lái). A standard "longest-match" algorithm will always pick "future," leading to a grammatical car crash in the translation. The paper seeks to identify these "sensitive" strings that are most likely to cause such failures.
Methods
The authors propose a systematic way to identify and handle these words:
- Defining the "Sensitive Word": They define it as a compound word where the components can naturally stand side-by-side in modern Chinese, but their individual roles are significantly different from the compound's role.
- Rule-Based Filtering: They established 18 word-formation patterns (e.g., Noun + Auxiliary Verb, Adverb + Transitive Verb) that are most likely to produce sensitive words.
- Dictionary Survey: They ran these patterns against a machine-readable dictionary of 87,600 items.
- Manual Verification: From the 12,357 words that matched the patterns, they manually verified which ones were truly "sensitive" in real-world usage.
- MT Strategy: They suggest that if an MT system fails to parse a sentence, it shouldn't just try every possible segmentation. Instead, it should specifically target and "break" any sensitive words first.
Key Insights and Ideas
- The "Longest Match" Trap: Most segmenters use a "greedy" approach, picking the longest possible word found in the dictionary. The authors show that for sensitive words, the longest match is often the wrong choice for the specific context.
- Not All Ambiguities are Equal: Breaking "Tokyo" (东京) into "East" and "Capital" is almost never correct in modern Chinese. However, breaking "Future" (将来) into "Will" and "Come" is a common requirement. The paper distinguishes between "non-sensitive" compounds (which should stay together) and "sensitive" ones (which are candidates for splitting).
- Syntactic Impact: The "sensitivity" of a word is directly tied to how much its split form changes the sentence structure. If splitting a word turns a Noun into a Verb + Verb sequence, it has a high "impact" on the translation.
Key Results
- Prevalence: Out of 87,600 words, 764 were identified as truly sensitive. While 0.9% seems small, these are often very common words, meaning they appear frequently in real text.
- High-Risk Patterns: Certain word structures are much more likely to be sensitive. For instance, 38.2% of words following the "Noun + Auxiliary Verb" (NC+VAUX) pattern were found to be sensitive.
- Efficiency Gain: By identifying these 764 words, MT developers can create a "re-segmentation priority list," significantly reducing the search space for alternative sentence structures when a primary parse fails.
Limitations and Future Work
- Manual Labor: The final identification of sensitive words relied on manual intuition, which is subjective and difficult to scale as language evolves.
- Incomplete Patterns: The authors admitted they didn't check all possible compositions (like Noun + Noun), suggesting the actual number of sensitive words is likely higher.
- Implementation Gap: The paper focuses on the survey and concept rather than providing a benchmark of the improved MT system's performance. The authors noted that the impact on MT efficiency would be reported in a later paper.
This research serves as a reminder that in NLP, "accuracy" on a sub-task is meaningless if it doesn't support the ultimate goal of understanding or translation. By defining "sensitivity," Ren and Nie provided a bridge between simple dictionary lookups and complex syntactic analysis.
Summary updated