PyToBI: A Toolkit for ToBI Labeling Under Python

Mónica Domínguez, Patrick Louis Rohrer, Juan Soler-CompanyInterspeech 2019

TL;DR

This paper introduces PyToBI, a Python-based toolkit for automatically annotating speech prosody using the ToBI (Tones and Break Indexes) convention. It aims to streamline the laborious process of prosodic annotation, making it more accessible for computational linguistics and NLP research.

What Problem Does This Paper Address?

Manually annotating speech prosody with conventions like ToBI is a time-consuming and perception-dependent task. While NLP tools are abundant in Python, there's a lack of accessible, ready-to-use toolkits for automatic prosodic annotation that can be easily integrated into existing Python-based NLP pipelines. This gap hinders cross-disciplinary research and the efficient compilation of prosodically annotated speech corpora.

Methods

PyToBI employs a two-stage processing pipeline. First, it leverages the established software Praat to extract acoustic parameters from audio files and their corresponding word/phone alignments (provided in TextGrid format). This includes identifying silences, intensity peaks and valleys, and extracting features like F0 slope, intensity range, and phone counts. The core innovation lies in the second stage, where these acoustic features are processed by a Python implementation. This Python module handles the automatic annotation of ToBI tones (pitch accents and boundary tones) and break indexes. The toolkit provides Python data structures for easy manipulation and export of the annotated information, and the output is also available in Praat's TextGrid format.

Datasets

The paper mentions an evaluation performed on an excerpt from a TED talk, annotated manually by two expert linguists. This excerpt comprised approximately 5 minutes of speech from a male American English speaker, resulting in 830 words annotated. The paper does not introduce a new benchmark dataset but rather uses a manually annotated corpus for evaluation.

Key Insights and Ideas

  • Bridging the NLP-Prosody Gap: PyToBI effectively integrates prosodic annotation capabilities into the Python ecosystem, which is dominant in NLP research.
  • Algorithmic Annotation: The toolkit offers an algorithmic approach to ToBI labeling, aiming to significantly speed up the annotation process compared to manual methods.
  • Unified ToBI Catalog: The system uses a reduced, unified catalog of ToBI labels for pitch accents and boundary tones, making it more manageable for automatic annotation while still allowing for parametric descriptions for further labels.
  • Praat Integration: PyToBI cleverly utilizes Praat for acoustic feature extraction, leveraging its robust capabilities while implementing the ToBI logic in Python for greater flexibility and integration.

Key Results

The evaluation showed promising results for PyToBI's automatic annotation capabilities:

  • Break Index Annotation: Achieved a high location precision of 0.97 and a label accuracy of 0.90.
  • Tone Annotation: Demonstrated a location precision of 0.77 and a label accuracy of 0.47.
  • Inter-annotator Agreement: The manual annotations by two linguists showed a location agreement of 0.91 and a label agreement of 0.78 for tones, and 1.00 location and 0.85 label agreement for breaks, providing a benchmark for the system's performance.

Limitations and Future Work

  • Tone Annotation Accuracy: The paper acknowledges that tone annotation accuracy (0.47 label score) is lower than break index annotation, suggesting room for improvement.
  • Language Specificity: The current unified catalog is primarily described in the context of English prosody.
  • Future Work: The authors plan to conduct a thorough evaluation across multiple languages and compare PyToBI with other existing prosody annotation tools like AuToBI and ANALOR. They also aim to expand the toolkit's capabilities for a wider range of ToBI labels.

Summary updated

BibTeX
@inproceedings{dominguez2019pytobi,
  author = {Mónica Domínguez and Patrick Louis Rohrer and Juan Soler-Company},
  title = {{PyToBI: A Toolkit for ToBI Labeling Under Python}},
  booktitle = {Interspeech},
  year = {2019},
}
Figure