Syllable-level prominence detection with acoustic evidence

Je Hun Jeon, Yang LiuInterspeech 2010 · OpenReview

Introduction

In spoken language, "prominence" refers to the emphasis or stress placed on certain syllables or words. Detecting this automatically is vital for applications ranging from high-quality text-to-speech synthesis to improving automatic speech recognition (ASR).

While many systems rely on lexical or syntactic information (the actual words spoken) to predict prominence, this approach fails when the underlying text transcript is inaccurate, such as in noisy ASR outputs. Consequently, researchers focus on acoustic evidence—the raw sound of the voice. Traditionally, this meant looking at pitch, energy, and duration. However, these features vary significantly between speakers. This paper investigates whether adding cepstral features (describing the spectral shape of the sound) and contextual information from neighboring syllables can make prominence detection more robust and accurate.

Methods

The authors framed prominence detection as a binary classification task: is a syllable prominent or not? They employed two main machine learning models: Multilayer Perceptrons (MLP) and Support Vector Machines (SVM).

Their feature engineering approach was multifaceted:

  • Prosodic Features: Pitch (F0), energy, and duration were extracted. Legendre polynomials modeled contours, and the Momel algorithm captured pitch slope changes.
  • Cepstral Features: Mel Frequency Cepstral Coefficients (MFCCs 1–12) were introduced to provide a stable spectral anchor.
  • Normalization: Z-score normalization was applied to pitch and energy to handle voice variation.
  • Contextual Expansion: Features were pulled from a window of up to two preceding ($L1, L2$) and two following ($R1, R2$) syllables.
  • Feature Selection: Backward elimination pruned the initial 121 features down to a highly effective set of 56.

Datasets

The study utilized the Boston University Radio News Corpus (BU), a benchmark dataset of news stories read by professional radio announcers. The corpus includes 253 utterances from four speakers (two male, two female). While originally annotated with complex ToBI (Tones and Break Indices) labels, the authors collapsed these into a binary "prominent vs. non-prominent" classification to ensure model robustness.

Key Insights: The Role of Context and Cepstral Features

A primary finding was that cepstral features act as "speaker levelers." While they performed worse than pitch or duration in isolation, they exhibited much lower variance across different speakers. This suggests they help the model identify syllable identity, providing a stable baseline regardless of the speaker's individual prosody.

Furthermore, the authors discovered a distinct "Look Back" Advantage. Context matters, but the preceding syllable ($L1$) proved significantly more informative than the following one ($R1$). The optimal performance was achieved using a window of two syllables back and one syllable forward ($L12R1$).

Impact of contextual syllable windows on classification performance.

Key Results

The system was evaluated in both Speaker Independent (SI) and Speaker Dependent (SD) setups. The best model achieved a state-of-the-art accuracy of 85.7% (SI) and 88.3% (SD), significantly outperforming previous benchmarks.

F-score performance across different feature sets and models.

As shown in the figure above, combining prosodic and cepstral features (ALL and SELECT sets) leads to a substantial increase in F-score compared to individual feature sets like Pitch (P) or Energy (E). The authors achieved a final F-score of 0.79 (SI) and 0.83 (SD), demonstrating a strong balance between precision and recall.

Limitations and Future Work

While successful, the study's reliance on the BU corpus means the approach has been tested primarily on professional announcers with clear prosody. Future work will explore how these acoustic-only models handle casual or emotional speech. Additionally, the authors plan to apply these feature expansion techniques to other tasks, such as phrasal boundary detection, and investigate semi-supervised learning to leverage larger, unannotated datasets.

Summary updated

BibTeX
@inproceedings{jeon2010syllable,
  author = {Je Hun Jeon and Yang Liu},
  title = {{Syllable-level prominence detection with acoustic evidence}},
  booktitle = {Interspeech},
  year = {2010},
  url = {https://openreview.net/forum?id=SRh1-qqA-ye},
}
Figure