Introduction
Code-switching (CS)—the practice of alternating between two or more languages within a single conversation—is a common feature of bilingual communication. Despite its prevalence, Automatic Speech Recognition (ASR) systems often struggle with CS because they are typically trained on "read speech" recorded in controlled environments. Spontaneous speech, however, is characterized by faster tempos, frequent fillers (e.g., "um," "ah"), and higher spectral variance.
To bridge this gap, the ASCEND (A Spontaneous Chinese-English Dataset) corpus provides 10.62 hours of high-quality, spontaneous bilingual dialogue. This dataset serves as a public benchmark for developing ASR models that can handle the complexities of real-world human interaction.
Dataset Characteristics
The ASCEND corpus was collected from informal, one-on-one conversations among 23 bilingual speakers. To ensure a diverse vocabulary, the researchers employed a session-based approach, prompting participants with topics ranging from technology and philosophy to personal hobbies.
The dataset is partitioned into training, validation, and test sets with disjoint speakers to ensure that models are evaluated on their ability to generalize to unseen voices. As shown in the figure above, the researchers maintained a relatively balanced gender distribution across these splits.
Methodology and Baselines
The authors established baselines using wav2vec 2.0, a self-supervised framework for speech representation learning. They fine-tuned three distinct versions of the model:
- Monolingual English: Pre-trained on Common Voice English.
- Monolingual Chinese: Pre-trained on Common Voice Chinese.
- Multilingual: Pre-trained on a corpus of 53 languages.
Performance was measured using Character Error Rate (CER) for Chinese and Mixed Error Rate (MER), which integrates CER for Chinese and Word Error Rate (WER) for English.
Analysis of Code-Switching
The researchers identified several key linguistic patterns within the ASCEND dataset:
- Language Turns: A "language turn" occurs when a speaker switches languages within a single utterance. In intra-sentential code-switching instances, speakers averaged approximately 2.18 turns per utterance.
- L1 Dominance: Despite the bilingual nature of the task, the speakers' first language (Chinese) remained dominant. Chinese segments were notably longer, averaging 8 characters, while English segments averaged 3 words.
- Topic Influence: Code-switching frequency was highly dependent on the subject matter. For example, "Technology" sessions exhibited the highest rate of intra-sentential switching (31.93%), often due to the use of English technical terminology within Chinese sentences.
Results and Discussion
The baseline experiments revealed that the Chinese-pre-trained wav2vec 2.0 model performed best, achieving a 22.69% CER and 27.05% MER on the test set. This suggests that because Chinese constitutes nearly half of the dataset, a monolingual model optimized for the dominant language's phonology can be more effective than a broader multilingual model.
Furthermore, the duration of monolingual utterances for both languages followed a similar distribution, peaking at 1–2 seconds. This consistency reflects the natural rhythm of spontaneous conversational turns.
Limitations and Future Directions
While ASCEND provides a valuable resource for the CS-ASR community, its current scale (10.62 hours) is modest compared to massive monolingual corpora like Librispeech. Future efforts will focus on expanding the dataset's volume and diversity, developing architectures that can better predict language switch points, and applying this data collection methodology to other language pairs.
Summary updated