TL;DR
This paper introduces a client-server speech translation system designed for mobile wireless clients, specifically PDAs connected via Wi-Fi, enabling bidirectional English-Japanese translation for travel conversations. By offloading heavy computation to powerful servers, the system achieves better translation accuracy and responsiveness than standalone devices, while overcoming the bandwidth limitations of traditional mobile networks.
What Problem Does This Paper Address?
The authors tackle the challenge of providing effective speech translation for travelers in foreign environments. While standalone speech translation devices are appealing for their portability, they are limited by the computational resources available on mobile hardware, leading to compromises in translation accuracy. Existing client-server approaches using mobile phones often suffer from narrow bandwidth telephone networks, which degrade speech recognition quality. This paper proposes a solution that leverages the increasing popularity and speed of wireless LAN (Wi-Fi) to enable high-quality speech translation on mobile devices like PDAs.
Methods
The system employs a client-server architecture. The client, a PDA running Windows CE, handles user interaction, voice input, and output. The server, running on Windows servers, performs the core speech translation tasks. Key components include:
- Speech Recognition: Large vocabulary continuous speech recognition for both English and Japanese, utilizing Hidden Markov Models (HMMs) and statistical language models. The system uses a two-stage search engine: word graph generation followed by optimal word sequence search.
- Machine Translation: A bi-directional translation module that accepts recognized expressions and outputs translated speech. It employs a Lexicalized Tree Automata-based Grammar (LTAM Grammar) formalism, which combines lexical and rule-based knowledge for handling colloquialisms and idiomatic expressions common in travel conversations.
- Communication: Voice input is digitized (16-bit monaural, 22 KHz sampling) and compressed using the G.722 algorithm to reduce the transmission rate to 88Kbps. This allows for more simultaneous clients and better performance on less stable connections. Communication between client and server is established via TCP/IP over an 802.11b wireless LAN.
Datasets
- Speech Recognition Corpora: Approximately 100K sentences for each language (Japanese and English) were used to train language models. These corpora specifically focus on travel conversations across various situations (hotel, restaurant, shopping, etc.).
- Machine Translation Dictionary: A combined lexicalized grammar dictionary was built. The Japanese-to-English dictionary contains about 150K words, and the English-to-Japanese dictionary contains about 70K words.
Key Insights and Ideas
- Wi-Fi as a Game Changer: The paper highlights wireless LAN as a crucial enabler for high-quality mobile client-server speech translation, offering sufficient bandwidth for rich audio data transmission, unlike traditional telephone networks.
- Client-Server Advantage: Offloading heavy computational tasks to powerful servers is a viable strategy to achieve superior speech translation accuracy and functionality on resource-constrained mobile devices.
- G.722 Compression Effectiveness: The G.722 speech compression algorithm, applied to raw audio data, significantly reduces bandwidth requirements without negatively impacting speech recognition accuracy, a key finding for practical mobile deployment.
- LTAM Grammar for Colloquialisms: The use of Lexicalized Tree Automata-based Grammar is a novel approach to effectively handle the nuances of colloquial and idiomatic expressions prevalent in travel conversations, which are often challenging for purely rule-based or statistical systems.
Key Results
- Speech Recognition Accuracy: Word accuracy of 97.5% for Japanese speakers and 92.0% for English speakers was achieved.
- Machine Translation Quality: A preliminary evaluation using a bilingual evaluator classified sentences into Good, Understandable, and Bad.
- Japanese-to-English: 88% of sentences were Good or Understandable; 66% were Good.
- English-to-Japanese: 90% of sentences were Good or Understandable; 74% were Good.
- Performance: The system demonstrated good performance in terms of speed and accuracy when tested on PC servers.
Limitations and Future Work
- Service Area Limitation: The primary limitation is the restricted service area dictated by wireless LAN availability. However, the authors suggest this is acceptable for specific environments like shops or buildings.
- PDA Microphone Quality: The built-in microphones on PDAs are noted as having inferior quality, though the system can accommodate external microphones, and future PDAs are expected to improve in this regard.
- Wireless Connection Reliability: The reliability of wireless connections remains an issue that needs further improvement.
- Future Work: The authors envision a future where ideal clients possess both standalone and client-server capabilities, allowing users to switch functions based on the situation. Further work will focus on integrating client operation and standalone operation compatibility.
Summary updated