TL;DR
This paper details the creation of the IIS_CSS, a novel spontaneous Chinese speech corpus designed to advance speaker-independent speech recognition and dialogue management for telephone-based systems. It outlines a robust data collection and annotation methodology, addressing the challenges of spontaneous speech disfluencies and real-world noise.
What Problem Does This Paper Address?
Developing robust automatic speech recognition (ASR) and dialogue management systems, particularly for spontaneous speech, requires large, diverse datasets. Existing corpora often lack the breadth of natural disfluencies, background noise, and dialectal variations found in real-world telephone conversations. This is especially true for Chinese spontaneous speech, where a large-scale, readily available corpus was missing. The authors tackle this by creating the IIS_CSS corpus, specifically for the CEST (Chinese-English Speech Translation) project, aiming to bridge this gap for speaker-independent ASR and dialogue management over telephone lines.
Methods
The IIS_CSS corpus collection employed a "dialogue-oriented" paradigm. Data was collected via telephone lines using Computerfone (CF) or Dialogue Card (DC) hardware, recording audio in Microsoft RIFF WAV format (16-bit, 8kHz). The collection involved two primary types of dialogues:
- Human-Human Dialogues: These were collected by recruiting callers (both internal and external volunteers) and having them interact with an "inside agent" (simulated or real).
- Human-Machine Dialogues: These were collected using a test spoken dialogue system built with a "Wizard of Oz" (WoZ) technique, allowing for more direct contribution to the development of the target spoken language system.
Data processing involved segmenting dialogues into utterances (averaging 3 seconds) using an adaptive silence detection algorithm and removing silence segments.
Annotation was a critical and challenging step due to the inherent variability of spontaneous speech. It was divided into two parts:
- General Information: This included dialect (ten types beyond Mandarin), speaker's mood, environment noise level (quantified into three levels), and speaker's sex.
- Textual Transcription: This involved transcribing spoken content. Special attention was paid to non-written language speech events, mispronunciations, and numbers. Non-speech events were compressed into seven symbols (detailed in Table 1). To ensure accuracy, a three-step labeling procedure was implemented, with each step performed by a different transcriber. A custom annotation tool was developed to streamline the process.
Datasets
The paper introduces the IIS_CSS corpus as a newly created benchmark. At the time of writing, the corpus contained:
- 10 hours of speech (with a plan to reach 40 hours).
- 350 calls.
- 4600 utterances.
- 147 speakers (93 male, 54 female).
The corpus consists of both human-human and human-machine dialogues, focusing on travel information retrieval and hotel front-desk services.
Key Insights and Ideas
- The "Dialogue-Oriented" Paradigm: This approach, combined with the WoZ technique for human-machine dialogues, proved effective in collecting naturalistic conversational data relevant to task-oriented systems.
- Addressing Spontaneous Speech Challenges: The paper highlights the significant effort required to develop a consistent and detailed criterion for annotating diverse spontaneous speech events, including disfluencies, mispronunciations, and non-speech sounds.
- Multi-faceted Annotation: The inclusion of general information (dialect, mood, noise) alongside textual transcription provides a richer dataset for studying robust ASR and understanding conversational context.
- The Value of a Large-Scale Chinese Corpus: The authors emphasize that the lack of such a corpus was a significant barrier to progress in large-vocabulary spontaneous Chinese speech recognition.
Key Results
- The first phase of the IIS_CSS corpus collection yielded 10 hours of speech and associated annotations.
- The corpus comprises both human-human (8 hours) and human-machine (2 hours) dialogues.
- A significant portion (85%) of the collected speech was spoken in an office-like environment, with only 5% falling into special dialects.
- The paper provides counts for various special events, such as filled pauses (5900), background noise (2300), and breath (98), illustrating the richness of the collected data.
Limitations and Future Work
Limitations:
- The corpus available at the time of publication was only one-fourth of the total foreseen requirement.
- While efforts were made to control for noise and dialect, the real-world collection environment meant these conditions could not be fully controlled.
Future Work:
- The corpus collection and transcription are planned to be completed by November 2000.
- The transcription will be expanded to serve translation purposes.
- The authors intend to continue collecting other domain-specific and close-speaking corpora.
Summary updated