Abstract
This paper presents our submission for the second automatic simultaneous translation workshop at NAACL 2021. We participated in both Chinese-to-English tracks: Chinese audio to English text and Chinese text to English text. Our approach utilizes a two-stage simultaneous translation pipeline composed of a QuartzNet ASR model and a BPE-based Transformer NMT engine. By employing rigorous data filtering and a wait-k inference strategy, we achieved a competitive BLEU score of 24.39 in the audio input track while effectively managing latency.
Introduction
Simultaneous translation (SimulMT) is a challenging task that requires translating speech in real-time, often before the source sentence is complete. This necessitates a delicate balance between latency (the delay experienced by the listener) and accuracy. Traditional systems often suffer from error propagation, where inaccuracies in Automatic Speech Recognition (ASR) negatively impact the Neural Machine Translation (NMT) output. Our system addresses these challenges through a robust pipeline designed to minimize errors and control latency for the Chinese-to-English translation task.
System Architecture
Our system adopts a two-stage pipeline architecture, allowing for independent optimization of the speech recognition and translation components.
Automatic Speech Recognition (ASR)
For the ASR component, we utilized QuartzNet15x5. This architecture is characterized by its use of depthwise separable 1D convolutions, which significantly reduces the number of parameters compared to models like Jasper while maintaining high accuracy. To handle long-form audio without the quadratic computational cost of standard self-attention, we integrated Memory-Self-Attention (MSA), ensuring linear complexity relative to input length.
Neural Machine Translation (NMT)
The NMT component is a standard Transformer model implemented using the Marian toolkit. We utilized a BPE-based approach for vocabulary management. The specific architectural configurations for the translation engine are detailed below:
Simultaneous Inference Policy
To manage the real-time nature of the task, we implemented the wait-k strategy. In this framework, the system waits for k source tokens before generating the first target token. Subsequently, it alternates between reading one source token and writing one target token. This provides a deterministic and controllable latency profile.
Data Engineering and Datasets
Data quality was a primary focus of our development. We employed several techniques to refine our training sets:
- ASR-Based Filtering: We used a pre-trained ASR model to transcribe our audio data and discarded samples where the transcript and audio alignment were poor.
- Cross-Entropy Filtering: Sentence pairs with low probability according to a pre-trained language model were removed.
- Back-Translation: We expanded our training set by generating synthetic Chinese source sentences from English corpora.
The ASR component was trained on a combination of public and proprietary Mandarin datasets to ensure robustness across domains:
For NMT, we primarily utilized the CWMT 19 corpus, incorporating both bilingual and monolingual data for training and back-translation.
Results and Discussion
Our pipeline demonstrated strong performance across the workshop's evaluation metrics:
- Audio Track: The system achieved a BLEU score of 24.39.
- Text-to-Text Track: We reached a BLEU score of 18.4 using the Consecutive Wait (CW) metric.
- ASR Accuracy: The ASR model achieved a Word Error Rate (WER) near 1.0 on our internal evaluation sets, providing a high-quality textual foundation for the translation stage.
By utilizing separable convolutions in QuartzNet, we maintained high efficiency, which is critical for minimizing the computational overhead in a simultaneous translation pipeline.
Conclusion and Future Work
While our two-stage pipeline achieved competitive results, we recognize the inherent limitations of independent component training, specifically the potential for a bottleneck where the NMT model is not optimized for ASR-specific errors. Future research will focus on end-to-end speech translation architectures. By integrating ASR and NMT into a single model, we aim to further reduce latency and preserve acoustic information, such as prosody and tone, that is currently lost during the intermediate text conversion phase.
Summary updated