TL;DR
This paper introduces a practical English-to-Japanese machine translation (MT) system designed for NHK's 24-hour satellite news broadcasts. It tackles the challenge of translating diverse news content under tight deadlines by employing a robust transfer-based MT approach, enhanced with local preprocessors for proper nouns and a sophisticated weighting mechanism for syntactic disambiguation.
What Problem Does This Paper Address?
Broadcasting international news often requires rapid translation for subtitles, a process traditionally handled by human translators working under immense time pressure. NHK's satellite news service, broadcasting in multiple languages, faces the challenge of providing Japanese subtitles for "World News" efficiently. The core problem is to automate or significantly assist this translation process to meet the demanding schedule and diverse linguistic content of global news. This involves handling a wide vocabulary, complex sentence structures, and specific linguistic features of news reporting, such as proper nouns and colloquialisms.
Methods
The system is built upon the STAR machine translation system and utilizes a transfer-based method, which involves morphological analysis, syntactic analysis, transfer, and generation. Key novelties and enhancements include:
- Local Context Translation (LOCT): A preprocessor that identifies and translates locally fixed sequences of words, particularly useful for human names with titles and other proper noun phrases. It uses a set of context-specific CFG rules distinct from the global analysis rules.
- Robust Processing of Undefined Words: To handle the vast and sometimes unpredictable vocabulary of news, the system employs strategies to estimate the grammatical roles of unknown words. This includes identifying local patterns (via LOCT), analyzing word endings (e.g., "-tion," "-ly"), and applying heuristic rules for capitalized words or numerals.
- Structural Disambiguation by Weighting Grammatical Rules: The syntactic analysis generates multiple possible sentence structures (AND/OR graph). A "weight mechanism" assigns costs to these structures, allowing the system to select the most probable interpretation. This mechanism is described as a search problem on an AND/OR graph where nodes have costs representing "incomprehensibility," "complexity," or "rareness."
- Word Selection by Semantic Markers: For Japanese word selection, semantic markers are used. This is particularly effective for disambiguating pronouns like "they" (distinguishing between animate and inanimate subjects) and for selecting specific translations of basic verbs like "catch" based on the semantic properties of their objects (e.g., using a
[CRIMINAL]marker).
Datasets
- Training/Development Data: The system was developed and tested using a large corpus of English news sentences from NHK's "World News" and AP (Associated Press) basic news service, totaling over 3.5 million words.
- Evaluation Data: The system was evaluated on 1,393 "World News" sentences during a three-month trial period.
Key Insights and Ideas
- The Importance of Contextual Preprocessing: The LOCT module highlights that for specific domains like news, specialized pre-processing for common patterns (like names and titles) can significantly improve translation accuracy, even before the main MT pipeline.
- Weighting as a Robust Disambiguation Strategy: The "weight mechanism" for syntactic analysis is crucial for handling the inherent ambiguity in natural language, especially in news where sentence structures can be complex or unusual. It provides a quantitative way to choose the "best" parse.
- Semantic Markers for Lexical Choice: Using semantic markers is an effective way to resolve lexical ambiguity in the target language (Japanese), particularly for words with multiple meanings or for pronoun resolution, by leveraging the semantic context of the sentence.
- News Language Specifics: The analysis of news sentences reveals distinct characteristics, such as a high frequency of proper nouns, human-subject verbs, and numeral expressions, which necessitate specialized handling in an MT system.
Key Results
- The weighting mechanism successfully selected the best candidates for approximately 78% of the analyzed "World News" sentences.
- Out of 1,393 "World News" sentences tested, 898 (64.5%) were successfully analyzed.
- Of the successfully analyzed sentences, 698 (78%) were properly translated as first candidates by the weight mechanism.
- Approximately 30% of analysis failures were attributed to errors in the input sentences (misspellings, grammar mistakes) and difficulties with colloquial expressions.
Limitations and Future Work
- Input Errors: The system is susceptible to errors in input sentences, such as misspellings and grammatical mistakes, which contributed significantly to analysis failures.
- Colloquial Expressions: Colloquial expressions were identified as a challenge and were difficult for the system to analyze.
- Numeral Expressions: The paper notes that numeral expressions, some of which are very complex, have not yet been fully considered and require improvement.
- Future Work: The authors plan to improve the system's treatment of numeral expressions and colloquial expressions. They are also designing a French-Japanese MT system with a similar structure.
Summary updated