Abstract: Simultaneous translation (ST) outputs the translation simultaneously while reading the input sentence, which is an important component of simultaneous interpretation. In this paper, we describe our submitted ST system, which won the first place in the streaming transcription input track of the Chinese-English translation task of AutoSimTrans 2021. Aiming at the robustness of ST, we first propose char-level simultaneous translation and applied wait-k policy on it. Meanwhile, we apply two data processing methods and combine two training methods for domain adaptation. Our method enhance the ST model with stronger robustness and domain adaptability. Experiments on streaming transcription show that our method outperforms the baseline at all latency, especially at low latency, the proposed method improves about 6 BLEU. Besides, ablation studies we conduct verify the effectiveness of each module in the proposed method.
Introduction
Simultaneous translation (ST) is a critical technology for real-time cross-lingual communication. Unlike traditional machine translation, ST must generate target text while the source input is still arriving. This "streaming" nature introduces significant challenges, particularly when the input comes from Automatic Speech Recognition (ASR) systems.
Existing systems typically rely on subword-level tokenization (like BPE). However, subword segmentation is inherently unstable in a streaming context; as new characters arrive, the segmentation of the entire prefix can change. Because ST systems generally cannot revise previous outputs, these segmentation shifts lead to irreversible errors. Furthermore, spoken language often lacks the formal punctuation found in the text corpora used for training, creating a domain mismatch that further degrades performance.
Methodology
To overcome these hurdles, we developed a system based on three primary pillars designed for robustness and adaptability.
1. Char-Level Wait-k Policy
We propose shifting the granularity of the source input from subwords to characters. In a "Wait-k" policy, the model waits for $k$ source tokens before it begins translating. By using characters as the fundamental unit:
- Stability is ensured: Characters are immutable. Unlike subwords, the prefix does not change as more input is received, eliminating re-segmentation errors.
- Latency is refined: Waiting for $k$ characters allows for much finer control over latency compared to waiting for $k$ subwords, which may contain many more characters.
2. Domain Adaptation and Data Processing
To handle the informal nature of spoken transcripts, we applied two key techniques:
- Depunctuation: We stripped ending punctuation from training sentences. This prevents the model from unnecessarily waiting for a period or question mark that may never appear in a continuous speech stream.
- Spoken Data Augmentation: We simulated ASR noise and spoken language characteristics by randomly adding modal particles, homophones, and character repetitions to our training data.
3. Advanced Training Techniques
We employed Multi-path training, allowing a single model to handle various latency requirements (different $k$ values) simultaneously. Additionally, we used Future-Guided training, where a full-sentence "teacher" model guides the simultaneous "student" model via knowledge distillation, helping the student better predict upcoming content from partial inputs.
Experimental Results
Our system was evaluated on the AutoSimTrans 2021 Chinese-English streaming transcription track. The model was pre-trained on the CWMT19 corpus (~9M pairs) and fine-tuned on the competition's spoken-language dataset (~38k pairs).
As shown in the results, the Char-Level Wait-k policy significantly outperformed the standard subword-level baseline. At a very low latency (Average Lagging ≈ 1.10), our method achieved a 6 BLEU point improvement. Ablation studies further highlighted the importance of our data processing: removing "Depunctuation" caused a 2.27 BLEU drop and increased latency by 2.83, proving that punctuation handling is vital for streaming performance.
Conclusion
By transitioning to character-level processing and implementing targeted domain adaptation, we developed a robust simultaneous translation system that effectively handles the instabilities of streaming ASR input. Our approach not only won first place in the AutoSimTrans 2021 competition but also demonstrates that character-level granularity is a superior foundation for low-latency, real-world simultaneous interpretation tasks.
Summary updated