RTTS: towards enterprise-level real-time speech transcription and translation services

Juan M. Huerta, Cheng Wu, Andrej Sakrajda, Sasha Caskey, Ea-Ee Jan, Alexander Faisman, Shai Ben-David, Wen Liu, Antonio Lee, Osamuyimen Stewart, Michael Frissora, David M. LubenskyInterspeech 2009 · DOI

Introduction

Real-Time Transcription and Translation Services (RTTS) represent an enterprise-grade architectural framework designed to bridge the gap between advanced speech and natural language processing (NLP) research and scalable business applications. Developed by IBM researchers, RTTS wraps complex speech recognition (ASR) and machine translation (MT) engines into a unified Web Services framework. This system is capable of powering diverse applications, ranging from real-time VoIP translation to instant messaging and high-volume batch transcription.

The Enterprise Challenge

While ASR and MT technologies have existed for decades, integrating them into a robust enterprise environment presents significant hurdles. Most legacy systems were either standalone portable devices or massive research frameworks not designed for multi-user, high-availability business needs. The authors identify four primary barriers to enterprise adoption:

  1. Resource Intensity: High-quality models require computational power far exceeding what mobile devices can provide.
  2. Protocol Fragmentation: Businesses utilize a heterogeneous mix of audio, text, and video across various protocols such as SIP and HTTP.
  3. Reliability: Systems must support load balancing and failover to ensure continuous service.
  4. Complexity: Developers should not require specialized linguistics expertise to integrate translation features into standard business applications.

System Architecture

The core of RTTS is a Web Services-based approach that encapsulates the complexities of ASR and MT behind a uniform interface. The architecture leverages the Unstructured Information Management Architecture (UIMA) to wrap engines as "annotators," which are managed by Java Message Service (JMS) to ensure reliable communication.

Detailed architecture of the RTTS system components.

Key components include:

  • Dual-Interface Access: Support for REST (text-based/batch requests) and SIP/RTP (real-time VoIP telephony).
  • Dialog Manager: The central logic unit coordinating resource allocation and session progress.
  • Efficiency Layers: To optimize processing speed, the system utilizes Translation Memories (human-verified pairs) and Translation Caches (storing previous machine outputs).

Optimization and User Experience

A critical finding in the research was the 65% Redundancy Rule: approximately 65% of content sent for translation was repetitive. By implementing a translation cache, the system could bypass heavy MT processing for these requests, reducing response times from 100ms to just 3ms.

Beyond backend efficiency, the researchers focused on user experience (UX) necessities for real-time interaction. Because ASR latency is tied to audio segment length, a "half-duplex" (push-to-talk) communication style was found to be more effective than full-duplex, as it helps manage user expectations regarding processing delays.

Real-time chat translation interface demonstrating the 'Back-Translation' feature.

To build user trust, the system incorporates Back-Translation. This provides a "round-trip" translation—translating the result back into the original language—allowing users to verify if the machine output is accurate or if they need to rephrase their input.

Key Results and Scalability

The RTTS framework demonstrated significant versatility across four distinct client types:

  1. VoIP Client: Real-time speech-to-speech translation over standard phone lines.
  2. SMS/Mobile Midlet: Asynchronous translation for mobile devices.
  3. Chat Plugin: Real-time "translate-as-you-type" functionality for instant messaging.
  4. Transcription Server: Batch processing for long-form media files like lectures and movies.

Performance metrics indicated that with a cache of only 20,000 entries, the system could fulfill 50% of all incoming requests without engaging the MT engine, significantly improving overall system throughput.

Conclusion

While RTTS successfully addresses scalability and protocol fragmentation, latency remains the primary challenge for real-time speech-to-speech systems. Future work focuses on further reducing system delays, improving automated error identification, and developing more robust recovery mechanisms for engine failures. By abstracting the complexities of NLP into a service-oriented architecture, RTTS provides a blueprint for the next generation of global enterprise communication tools.

Summary updated

BibTeX
@inproceedings{huerta2009rtts,
  author = {Juan M. Huerta and Cheng Wu and Andrej Sakrajda and Sasha Caskey and Ea-Ee Jan and Alexander Faisman and Shai Ben-David and Wen Liu and Antonio Lee and Osamuyimen Stewart and Michael Frissora and David M. Lubensky},
  title = {{RTTS: towards enterprise-level real-time speech transcription and translation services}},
  booktitle = {Interspeech},
  year = {2009},
  doi = {10.21437/interspeech.2009-157},
}
Figure