TL;DR
This paper introduces a mobile application for Chinese-Spanish statistical machine translation (SMT) that goes beyond simple text input. It integrates multiple input methods like speech and image-based OCR, offers offline translation capabilities, and aims to improve user experience through features like language detection and image retrieval.
What Problem Does This Paper Address?
While statistical machine translation (SMT) has matured, deploying it effectively on mobile devices for less common language pairs like Chinese-Spanish presents challenges. Traditional SMT systems often require constant internet connectivity, which can be unreliable on mobile. Furthermore, user interaction with translation apps can be cumbersome, especially when dealing with complex characters or when users need to quickly understand visual information. This paper addresses the need for a more versatile, user-friendly, and accessible Chinese-Spanish translation solution on mobile platforms, incorporating diverse input methods and offline functionality.
Methods
The core of the system is a direct Chinese-Spanish statistical machine translation (SMT) engine. Unlike many systems that rely on a pivot language (like English) for translation between distant language pairs, this system was trained directly on a substantial Chinese-Spanish corpus. This direct approach is a key differentiator.
The mobile application itself is built using native development tools for Android (ADT with Eclipse IDE) and iOS (Xcode). Communication between the mobile client and the server-side SMT engine is handled via HTTP POST requests.
Key features and their underlying methods include:
- Multiple Input Modalities:
- Text Input: Standard text entry.
- Automatic Speech Recognition (ASR): Leverages native ASR engines of the mobile platforms (Jelly Bean for Android, Siri for iOS).
- Optical Character Recognition (OCR): Utilizes the open-source Tesseract OCR engine, adapted for the application.
- Language Detection: A simple, efficient algorithm based on comparing code-average values with a numeric threshold for UTF-8 encoding, allowing the app to automatically identify Chinese or Spanish input.
- Image Retrieval: Integrates with Flickr to fetch relevant images for translated concepts or entities, aiding in understanding specific terminology.
- Offline Mode: Implements a basic index-based search strategy. It stores frequently used translations locally, allowing for semantic similarity searches when the device is offline. This index is periodically updated when the app is online.
Datasets
The paper highlights the creation of a substantial Chinese-Spanish parallel corpus for training the direct SMT engine. This corpus is a combination of several resources:
- The Holy Bible corpus
- The United Nations corpus
- A subset of European Parliament Plenary Speeches (automatically translated into Chinese)
- A technical Chinese-Spanish parallel corpus from TAUS
- A small in-house developed corpus in transportation and hospitality domains
The total size of this combined corpus is 70 million words.
For language-specific preprocessing, the following tools were used:
- Chinese: Stanford segmenter
- Spanish: Freeling
The SMT engine itself was trained using the MOSES toolkit with standard configurations for alignment, language modeling (5-gram with interpolation and Kneser-Ney discount), and reordering. An in-house corpus was used for optimization.
Key Insights and Ideas
- Direct Translation for Less Common Pairs: Training a direct Chinese-Spanish SMT model, rather than relying on a pivot language, is feasible and potentially more effective when sufficient parallel data can be curated. This bypasses potential error propagation from intermediate translations.
- Multimodal Input is Crucial for Mobile: Integrating ASR and OCR significantly enhances the usability of translation apps on mobile devices, catering to diverse user needs and contexts beyond simple typing.
- Offline Functionality is a Game-Changer: Providing basic offline translation capabilities, even if based on a simple index, addresses a critical limitation of many online-dependent mobile applications, especially in areas with poor connectivity.
- Visual Aids Enhance Understanding: Image retrieval for translated terms can be more effective than pure text for specific types of vocabulary (e.g., local names, food), bridging semantic gaps.
- User Feedback Loop for Improvement: Incorporating a mechanism for users to suggest better translations offers a clear path for continuous improvement and adaptation of the SMT engine over time.
Key Results
The paper focuses on the system's architecture and features rather than presenting extensive quantitative evaluation metrics. However, it states that the direct Chinese-Spanish SMT system was trained on a 70 million word corpus. The application successfully integrates:
- Native ASR and OCR capabilities.
- An automatic language detection system.
- Image retrieval functionality.
- Basic offline translation capabilities through an index-based search.
The authors emphasize the system's ability to provide a more convenient and accessible translation experience on mobile platforms.
Limitations and Future Work
- Offline Mode Simplicity: The current offline mode relies on a simple index-based search and semantic similarity. The authors acknowledge this is a basic implementation.
- User Suggestion Integration: While a mechanism for users to suggest better translations is planned, its implementation and impact on re-training the SMT engine are future work.
- Continuous Improvement: The authors plan to use the collected user suggestions to re-train the main SMT engine, aiming to improve its coverage and overall quality.
- Further Enhancement Plans: Beyond user suggestions, future work could involve more sophisticated offline translation strategies and further integration of diverse input and output modalities.
Summary updated