SEAME: a Mandarin-English code-switching speech corpus in south-east asia

Dau-Cheng Lyu, Tien Ping Tan, Engsiong Chng, Haizhou Li

Abstract

This paper introduces SEAME (South-East Asia Mandarin-English), a 30-hour spontaneous speech corpus specifically designed to capture the linguistic nuances of code-switching in Singapore and Malaysia. Unlike previous datasets that relied on read speech or artificial segments, SEAME provides high-quality, time-aligned transcriptions of natural discourse. This resource is essential for training Automatic Speech Recognition (ASR) and Language Identification (LID) systems to handle the complex, multi-lingual reality of South-East Asian communication.

Introduction

In multilingual societies like Singapore and Malaysia, speakers frequently engage in intra-sentential code-switching—the practice of mixing two or more languages within a single sentence. While natural for local speakers, this phenomenon presents a significant challenge for traditional Large Vocabulary Continuous Speech Recognition (LVCSR) systems, which are typically optimized for monolingual input. To build robust models, researchers require data that reflects spontaneous, real-world talk rather than scripted or artificial speech.

Methodology

The researchers collected 120 hours of raw audio, distilling it into a 30-hour gold-standard corpus. The methodology focused on:

  • Spontaneous Elicitation: Using interviews and informal conversations. Interviewers used bilingual questions to naturally trigger code-switching responses.
  • Multi-Tiered Annotation: Using the ELAN tool, speech was categorized into six types, including target code-switched speech, particles (e.g., ah, lah), other dialects (e.g., Hokkien), and non-speech signals.
  • Precise Alignment: A combination of force-alignment and manual adjustment via WaveSurfer ensured language boundaries were accurate to the millisecond.
Figure 2: Example of a code-switched utterance showing waveform, spectrogram, and time-aligned transcriptions for English and Mandarin segments.

The SEAME Corpus

The resulting SEAME corpus consists of 30 hours of transcribed data, split equally between Singaporean and Malaysian speakers. The dataset features 97 speakers, primarily college students, recorded at 16kHz. While the corpus is Mandarin-dominant, it contains a high density of English switches, providing 25,000 intra-sentential code-switching utterances for model training.

Linguistic Insights and Results

Analysis of the SEAME corpus revealed several key characteristics of South-East Asian code-switching:

The "One-Word" Switch

A striking finding was the prevalence of very short switches. As shown in the data, a significant majority of English switches consisted of just a single word—approximately 50% for Singaporean speakers (represented by NTU) and 70% for Malaysian speakers (represented by USM). This suggests that code-switching often serves as a lexical "fill-in" for specific terms.

Figure 4: Distribution of word counts per language switch, highlighting the dominance of single-word switches in both Singaporean (NTU) and Malaysian (USM) data.

Contextual and Structural Patterns

  • Environmental Influence: Data showed a sharp divide between home and campus language use. For instance, while 68% of Singaporean speakers used Mandarin primarily at home, 70% switched to English as their primary language on campus.
  • Language "Stickiness": Mandarin segments tended to be longer and more grammatically complex than English switches within the same utterance.
  • Turn Frequency: Code-switched utterances averaged 2.8 to 3.1 language turns within just a few seconds, illustrating the high frequency of switching in spontaneous talk.

Conclusion and Future Work

The SEAME corpus provides a dense resource for training Language Boundary Detection (LBD) and bilingual acoustic models. While the current 30-hour set is Mandarin-dominant, it establishes a foundation for future work. The authors intend to expand the dataset and explore how lexical triggers (e.g., "I think" or "我就") and prosodic patterns can be used to predict language switching in real-time systems.

Summary updated

BibTeX
@inproceedings{lyu2010seame,
  author = {Dau-Cheng Lyu and Tien Ping Tan and Engsiong Chng and Haizhou Li},
  title = {{SEAME: a Mandarin-English code-switching speech corpus in south-east asia}},
  booktitle = {Interspeech},
  year = {2010},
  doi = {10.21437/interspeech.2010-563},
}
Figure