Data collection and processing in a Chinese spontaneous speech corpus IIS_CSS

Junlan Feng, Xianfang Wang, Limin Du

TL;DR

This paper details the creation of the IIS_CSS, a novel spontaneous Chinese speech corpus designed to advance speaker-independent speech recognition and dialogue management for telephone-based systems. It outlines a robust data collection and annotation methodology, addressing the challenges of spontaneous speech disfluencies and real-world noise.

What Problem Does This Paper Address?

Developing robust automatic speech recognition (ASR) and dialogue management systems, particularly for spontaneous speech, requires large, diverse datasets. Existing corpora often lack the breadth of natural disfluencies, background noise, and dialectal variations found in real-world telephone conversations. This is especially true for Chinese spontaneous speech, where a large-scale, readily available corpus was missing. The authors tackle this by creating the IIS_CSS corpus, specifically for the CEST (Chinese-English Speech Translation) project, aiming to bridge this gap for speaker-independent ASR and dialogue management over telephone lines.

Methods

The IIS_CSS corpus collection employed a "dialogue-oriented" paradigm. Data was collected via telephone lines using Computerfone (CF) or Dialogue Card (DC) hardware, recording audio in Microsoft RIFF WAV format (16-bit, 8kHz). The collection involved two primary types of dialogues:

  • Human-Human Dialogues: These were collected by recruiting callers (both internal and external volunteers) and having them interact with an "inside agent" (simulated or real).
  • Human-Machine Dialogues: These were collected using a test spoken dialogue system built with a "Wizard of Oz" (WoZ) technique, allowing for more direct contribution to the development of the target spoken language system.

Data processing involved segmenting dialogues into utterances (averaging 3 seconds) using an adaptive silence detection algorithm and removing silence segments.

Annotation was a critical and challenging step due to the inherent variability of spontaneous speech. It was divided into two parts:

  1. General Information: This included dialect (ten types beyond Mandarin), speaker's mood, environment noise level (quantified into three levels), and speaker's sex.
  2. Textual Transcription: This involved transcribing spoken content. Special attention was paid to non-written language speech events, mispronunciations, and numbers. Non-speech events were compressed into seven symbols (detailed in Table 1). To ensure accuracy, a three-step labeling procedure was implemented, with each step performed by a different transcriber. A custom annotation tool was developed to streamline the process.

Datasets

The paper introduces the IIS_CSS corpus as a newly created benchmark. At the time of writing, the corpus contained:

  • 10 hours of speech (with a plan to reach 40 hours).
  • 350 calls.
  • 4600 utterances.
  • 147 speakers (93 male, 54 female).

The corpus consists of both human-human and human-machine dialogues, focusing on travel information retrieval and hotel front-desk services.

Key Insights and Ideas

  • The "Dialogue-Oriented" Paradigm: This approach, combined with the WoZ technique for human-machine dialogues, proved effective in collecting naturalistic conversational data relevant to task-oriented systems.
  • Addressing Spontaneous Speech Challenges: The paper highlights the significant effort required to develop a consistent and detailed criterion for annotating diverse spontaneous speech events, including disfluencies, mispronunciations, and non-speech sounds.
  • Multi-faceted Annotation: The inclusion of general information (dialect, mood, noise) alongside textual transcription provides a richer dataset for studying robust ASR and understanding conversational context.
  • The Value of a Large-Scale Chinese Corpus: The authors emphasize that the lack of such a corpus was a significant barrier to progress in large-vocabulary spontaneous Chinese speech recognition.

Key Results

  • The first phase of the IIS_CSS corpus collection yielded 10 hours of speech and associated annotations.
  • The corpus comprises both human-human (8 hours) and human-machine (2 hours) dialogues.
  • A significant portion (85%) of the collected speech was spoken in an office-like environment, with only 5% falling into special dialects.
  • The paper provides counts for various special events, such as filled pauses (5900), background noise (2300), and breath (98), illustrating the richness of the collected data.

Limitations and Future Work

Limitations:

  • The corpus available at the time of publication was only one-fourth of the total foreseen requirement.
  • While efforts were made to control for noise and dialect, the real-world collection environment meant these conditions could not be fully controlled.

Future Work:

  • The corpus collection and transcription are planned to be completed by November 2000.
  • The transcription will be expanded to serve translation purposes.
  • The authors intend to continue collecting other domain-specific and close-speaking corpora.

Summary updated

Abstract

In this paper we report on the first phase of the speech corpus ISS_CSS collection for purposes of the CEST(Chinese-English speech translation) project. The corpus is intended to provide training material for speaker independent spontaneous Chinese speech recognition and automatic dialogue management over the telephone line. This paper describes the collection measures, processing methods, annotation and contents of this corpus. It consists of two parts: human-human dialogues and human-machine dialogues. Presently, the corpus has finished 10-hour speech and the associated annotation. Finally, we will present our collecting plan in the future.
BibTeX
@inproceedings{feng2000data,
  author = {Junlan Feng and Xianfang Wang and Limin Du},
  title = {{Data collection and processing in a Chinese spontaneous speech corpus IIS\_CSS}},
  booktitle = {Interspeech},
  year = {2000},
  doi = {10.21437/icslp.2000-558},
}
Figure