Introduction
Automatic Speech Recognition (ASR) systems frequently struggle in multilingual contexts where speakers engage in "code-switching"—the practice of alternating between two or more languages within a single conversation (e.g., "我要 start on 我的 essay"). Correcting these errors is a critical post-processing step; however, the field faces a significant data bottleneck. High-quality, human-annotated parallel data for code-switching is scarce and expensive to produce. This paper addresses this challenge by introducing two new benchmarks and a novel data augmentation strategy.
New Benchmarks: SEAME-C and ASCEND-C
To facilitate research, the authors present two new datasets derived from existing corpora:
- SEAME-C: Based on the SEAME corpus (Singapore/Malaysia), this dataset is characterized by high code-switching density, with 76.6% of sentences containing mixed languages.
- ASCEND-C: Based on the ASCEND corpus (Hong Kong), this dataset is more Chinese-dominant, with code-switching appearing in 27.8% of sentences.
Analysis of these datasets reveals that "Word Selection" errors (often involving homophones) constitute the vast majority (81-85%) of ASR mistakes in code-switching scenarios, providing a clear target for correction models.
Methodology
The proposed solution involves a specialized model architecture paired with a novel data generation technique.
Sequence Editing Model
Rather than using standard sequence-to-sequence models that regenerate entire sentences, the authors employ a Sequence Editing Model based on m-BERT. This model predicts specific edit operations—keep, delete, append, or replace—for each token, making it highly efficient for error correction where the majority of the input is typically correct.
Controllable Iterative (CI) Data Augmentation
The core contribution is the Controllable Iterative (CI) data augmentation method. This process generates high-quality training pairs from monolingual text through an iterative "unfolding" process.
As shown in Figure 1, the process begins by transforming a monolingual sentence into a "clean" code-switched target. Subsequent iterations introduce synthetic ASR noise (e.g., homophone swaps). A "Confidence Bias" parameter allows researchers to control the intensity of the generated errors, while a BERTScore-based filter ensures the resulting pairs remain semantically coherent.
Experimental Results
The CI method was evaluated using the MaxMatch ($M^2$) scorer and Mixed Error Rate (MER).
Performance Gains
On the SEAME-C dataset, the CI-augmented mT5 model achieved an $F_{0.5}$ score of 26.5, significantly outperforming rule-based (25.4) and back-translation (25.5) baselines. Notably, specialized models fine-tuned on this data even outperformed large language models like ChatGPT (GPT-3.5-turbo), highlighting the value of task-specific training data.
The Iteration Bottleneck
A key finding of the study is the relationship between the number of augmentation iterations and model performance.
As illustrated in Figure 2, while increasing iterations initially improves performance by providing more diverse training signals, the quality of generated errors begins to degrade after the 3rd iteration. This "3-iteration sweet spot" represents the optimal balance between data volume and noise quality.
Conclusion
This work demonstrates that controllable, iterative data augmentation can effectively overcome the data scarcity hurdle in code-switching ASR error correction. By combining specialized sequence editing models with high-quality synthetic data, the authors achieve state-of-the-art results that surpass general-purpose LLMs. Future work will explore extending this methodology to other language pairs and low-resource linguistic tasks.
Summary updated