Structure and annotation of Polish LVCSR speech database

Katarzyna Klessa, Grażyna DemenkoInterspeech 2009

TL;DR

This paper introduces JURISDIC, a large-scale, meticulously annotated Polish speech database designed for Large Vocabulary Continuous Speech Recognition (LVCSR). It emphasizes the critical role of rich linguistic and acoustic-phonetic annotation for improving LVCSR models, moving beyond simple recording statistics.

What Problem Does This Paper Address?

Building robust LVCSR systems, especially for less-resourced languages like Polish, hinges on the availability of large, high-quality speech databases. However, simply collecting hours of speech isn't enough. The authors highlight the challenge of creating databases with detailed linguistic and acoustic-phonetic annotations that can be effectively leveraged to improve acoustic models. They argue that formal evaluation should extend beyond recording statistics to encompass the linguistic structures and annotation quality itself, a gap they aim to fill with their JURISDIC database.

Methods

The JURISDIC database was constructed with specific principles in mind: low redundancy, acoustic-phonetic variability, representativeness, and a balanced, heterogeneous structure. The annotation process involved a multi-stage approach:

  • Orthographic Transcription: Initial word-level transcription.
  • Manual Validation: Annotators (students) validated transcriptions, inserted special event markers, and defined time boundaries.
  • Expert Review: Hand-validation by expert phoneticians and experienced labelers, with inter-labeler agreement monitored.
  • Automatic Segmentation: Phone-level labeling using an integrated tool (Salian), with manual validation planned for the future.

The database structure incorporates detailed metadata stored in a SQL database, managed by a custom client-server application.

Datasets

The primary dataset introduced is the JURISDIC Polish speech database. It comprises over 1500 annotated sessions from 16 regions of Poland, with an additional 500 experimental recordings, totaling over 1155 hours of speech from 2137 speakers. The database is structured into three main sub-corpora:

  • "Police&Office": Read and semi-spontaneous speech, further divided into types for phonetic coverage, dictation, and specialized vocabulary.
  • "Court": Spontaneous speech recorded in real court trials.
  • "Lawyer": Read speech recorded in lawyers' offices, focusing on specialized legal language.

Key Insights and Ideas

  • Annotation is King: The paper strongly advocates for detailed linguistic and acoustic-phonetic annotation as crucial for improving LVCSR performance, arguing it's as important as recording statistics.
  • Beyond Basic Counts: Simply having large amounts of data isn't sufficient; the quality and richness of the annotation are key differentiators.
  • Speaker Strategies Matter: The database structure allows for analysis of speaker strategies (e.g., use of fillers, hesitations) based on task type (read vs. spontaneous speech), which can inform acoustic modeling.
  • Language Specificity is Crucial: The database design and annotation specifications are tailored to the peculiarities of the Polish language, including its phonetic and grammatical features.
  • Formal Evaluation of Annotations: The authors propose that formal evaluation procedures should apply not only to recording statistics but also to the linguistic structures and annotation quality.

Key Results

  • The JURISDIC database contains over 5.2 million words in its annotation files.
  • Analysis of special events revealed that spontaneous speech (court recordings) had significantly more fillers than read speech.
  • Speaker noises were higher in domain-specific read texts (especially "Lawyer") compared to other text types.
  • Mispronunciation and unintelligible speech markers were highest in court recordings and specialized read texts, suggesting these are more challenging for speakers and/or annotators.
  • Preliminary acoustic modeling results using JURISDIC data showed promise in incorporating accent and utterance speech rate information.

Limitations and Future Work

  • Manual Validation of Phone-Level Segmentation: While automatic phone-level segmentation was performed using Salian, full manual validation is planned for the future.
  • Inter-Labeler Agreement Nuances: While generally high, inter-labeler agreement for certain special event labels (speaker noise, mispronunciation) was lower, attributed partly to variations in annotator judgment.
  • Future Work:
    • Further examination of the obtained data.
  • Sophisticated, language-oriented parsing of annotation data to extract linguistic factors.
  • Development of tools for semi-automatic evaluation of annotations (random selection for hand-validation, comparison with reference lexicons).
  • Analysis of the results from comparing annotation word lists with reference lexicons.
  • Continued exploration of speaker identification and characterization.
  • Studies on rhythm modeling, pronunciation differences, and segmental/suprasegmental variation.

Summary updated

BibTeX
@inproceedings{klessa2009structure,
  author = {Katarzyna Klessa and Grażyna Demenko},
  title = {{Structure and annotation of Polish LVCSR speech database}},
  booktitle = {Interspeech},
  year = {2009},
}
Figure