TL;DR
This paper tackles the challenge of automatically translating Chinese signs into English for tourists. The authors present a system that detects, recognizes, and translates signs, comparing two data-driven machine translation approaches: Example-Based Machine Translation (EBMT) and Statistical Machine Translation (SMT). They find that EBMT is more accurate for signs seen in training data, while SMT shows better generalization, suggesting a multi-engine approach could be optimal.
What Problem Does This Paper Address?
Navigating foreign countries can be challenging, especially when signs contain crucial information that isn't understood. This paper focuses on the specific problem of translating signs found in natural environments, such as street signs, warnings, and commercial advertisements, from Chinese to English. The authors highlight that signs present unique translation difficulties compared to standard text due to their conciseness, frequent use of abbreviations and idioms, and potential for ungrammatical language. This is particularly relevant for tourists who may not have access to immediate translation assistance.
Methods
The core of the system involves three key technologies: sign detection, optical character recognition (OCR), and language translation. The paper's primary contribution lies in its comparison of two data-driven machine translation (DDMT) methods for the translation module:
- Example-Based Machine Translation (EBMT): This approach relies on a bilingual corpus of sign examples. When a new sign is encountered, the system searches for similar examples in its corpus to generate a translation. It uses a shallow matching system and can extract bilingual dictionaries statistically.
- Statistical Machine Translation (SMT): This method models translation as a noise channel problem, inferring statistical models (language and translation models) from bilingual corpora. The authors specifically discuss a modified IBM Model-1, enhanced with a bi-directional training process and a bilingual glossary to improve alignment accuracy, especially with limited training data.
The system architecture is described as a pipeline: a video camera captures images, a "Detection & Recognition & Translation Module" processes them, and an "Interactive Module" handles user input and output.
Datasets
The paper utilizes a small bilingual sign corpus for training:
- Approximately 670 Chinese signs with translations from "Read Chinese Signs" by Kubler (1993).
- An additional 1300 signs translated from photos taken in China by native speakers.
This combined corpus of roughly 2000 parallel sign sentences is noted as being very small for data-driven systems. For SMT training, a Chinese-English glossary from the Language Data Consortium (LDC) containing about 128,366 entries was also used. The evaluation was performed on a set of 70 test signs.
Key Insights and Ideas
- Signs are a distinct genre: The authors emphasize that sign language is different from formal written text, posing unique challenges for MT systems due to abbreviations, idioms, and conciseness leading to ambiguity.
- Data-driven approaches are promising but data-hungry: Both EBMT and SMT are presented as viable alternatives to knowledge-based systems for sign translation, as they don't require extensive manual rule creation. However, their performance is significantly impacted by the limited size of the available bilingual sign corpus.
- EBMT excels at direct matches, SMT at generalization: When a sign is present in the training data, EBMT tends to produce more correct translations. Conversely, SMT, with its statistical modeling, shows a greater ability to infer patterns and handle unseen signs, leading to more "partially correct" translations.
- Bi-directional training and glossaries boost SMT with small data: The modified SMT approach, incorporating bi-directional alignment and a bilingual glossary, proves effective in improving translation quality even with a very small training set.
- Segmentation is a critical bottleneck for EBMT: Errors in Chinese word segmentation disproportionately affect EBMT, as it hinders the crucial source-language matching process.
Key Results
The paper reports on the evaluation of EBMT and SMT on 70 test signs, categorizing translations as "Correct," "Partial Correct," or "Wrong."
- EBMT: Achieved 31 (44%) "Correct" translations, 21 (30%) "Partial Correct," and 18 (26%) "Wrong."
- SMT: Achieved 22 (31%) "Correct" translations, 28 (40%) "Partial Correct," and 20 (29%) "Wrong."
While EBMT had a higher percentage of fully correct translations, SMT produced more partially correct ones, indicating its better generalization capabilities. The analysis of error sources showed that segmentation errors were a larger problem for EBMT, while both systems suffered from the small training data size.
Limitations and Future Work
- Small Training Data: The most significant limitation acknowledged is the small size of the bilingual sign corpus, which impacts the performance of both DDMT methods.
- Evaluation Method: The authors opted for manual labeling ("Correct," "Partially Correct," "Wrong") over standard n-gram-based metrics like BLEU or NIST, arguing that these metrics are less suitable for short sign sentences and don't fully capture the practical impact on user understanding.
- Future Work: The authors are actively working on developing a multi-engine machine translation system. This system aims to incrementally learn from data and combine the strengths of different translation engines (like EBMT and SMT) to achieve better overall performance. They also suggest that introducing generalization approaches like word clustering and grammar/template induction could help alleviate the data scarcity issue.
Summary updated