Acoustic correlates of perceived syllable prominence in German

Hansjörg Mixdorff, Christian Cossio-Mercado, Angelika Hönemann, Jorge Gurlekian, Diego Evin, Humberto TorresInterspeech 2015

TL;DR

This paper investigates how acoustic features of speech, like pitch and duration, relate to perceived syllable prominence in German. The authors found that while acoustic cues strongly correlate with prominence, accurately identifying sentence focus (a key aspect of meaning) from these cues alone remains challenging for human listeners, suggesting a need for more sophisticated models in speech technology.

What Problem Does This Paper Address?

Understanding the relationship between the acoustic properties of speech and its underlying meaning is crucial for advancing technologies like automatic speech recognition (ASR) and pronunciation training. Specifically, this paper tackles the challenge of how syllable prominence, a key prosodic feature, is acoustically realized and how it reflects linguistic information such as sentence modality (e.g., statement vs. question) and focus (which part of the sentence is emphasized). Current ASR systems often underutilize prosodic information, limiting their ability to truly understand spoken language. This research aims to bridge this gap by exploring the acoustic correlates of perceived prominence and its connection to linguistic meaning, with the ultimate goal of enabling ASR systems to better interpret these nuances.

Methods

The study employed a two-pronged approach: acoustic analysis of recorded speech and a perceptual experiment.

For the acoustic analysis, German sentences were recorded from native speakers under various focus and modality conditions. These recordings were then meticulously segmented at the syllable level. The researchers utilized the Fujisaki model to parameterize the fundamental frequency (F0) contours, extracting accent command amplitudes (Aa) as a measure of F0 gesture magnitude. Additionally, they analyzed syllable durations, maximum intensity, and mean harmonics-to-noise ratio (HNR).

The perceptual experiment involved human listeners evaluating the same recorded utterances. Subjects were tasked with identifying the sentence modality and the intended focus. Crucially, they also rated the prominence of each syllable using a slider-based interface, which was then mapped to an integer scale. The researchers then correlated these perceptual ratings with the acoustic features extracted from the same utterances.

Datasets

The study used a corpus of six short German sentences, originally created by Andreeva et al. for studies on focus distinctions. This corpus was expanded by recording ten additional speakers at Beuth University, including sentences in interrogative mode. The recordings were down-sampled to 16 kHz and force-aligned using the WEVOSYS LINGWAVES aligner. A total of 528 utterances were selected for the perception experiment after auditory checks for appropriate focus type and sentence mode.

Key Insights and Ideas

  • Strong Acoustic-Prominence Link: Perceived syllable prominence is strongly correlated with acoustic features, particularly the amplitude of underlying accent commands (Aa), syllable duration, maximum intensity, and mean harmonics-to-noise ratio.
  • Focus vs. Modality: While acoustic features clearly differentiate between statements and questions (especially with high F0 plateaus and question-final rises), distinguishing between different types of focus (broad vs. narrow, contrastive vs. non-contrastive) based solely on acoustics is more challenging.
  • Broad Focus Ambiguity: Broadly focused utterances were acoustically distinguishable from narrow focus, but human listeners struggled to correctly identify them, often mistaking them for non-contrastively focused utterances.
  • Prominence Prediction: A regression model incorporating acoustic features like Aa, duration, and intensity could explain a significant portion (76.1%) of the variance in perceived prominence.
  • HNR and Questions: The mean harmonics-to-noise ratio showed a significant correlation with prominence, and a notable drop in HNR was observed in focused items of echo-questions, potentially reflecting a specific vocal quality associated with incredulity.

Key Results

  • Focus Identification Accuracy: Questions were correctly identified in 94% of judgments. However, identifying the focus condition of statement utterances was less accurate, with broad focus being identified correctly in only about one-third of cases.
  • Prominence Prediction Model: A regression model explained 76.1% of the variance in perceptual syllable prominence. Adding base frequency (Fb) slightly increased this to 76.4%.
  • Acoustic Feature Correlations: Perceived prominence showed strong correlations with Aa (r=0.738), moderate correlations with duration (r=0.418) and max intensity (r=0.369), and a weaker correlation with mean HNR (r=0.317).
  • Focus Width Effects: Focus width significantly affected F0, duration, and intensity, but not the mean HNR. Contrastive vs. non-contrastive focus distinctions were less clear acoustically.

Limitations and Future Work

The authors acknowledge that identifying broad focus from acoustic cues alone is difficult, even for human listeners. The perceptual experiment's reliance on subjects choosing the most appropriate question to identify focus might have introduced bias. The study also notes that the slider-based prominence assessment might not perfectly capture all nuances, especially with the strong influence of F0.

Future work includes comparing these findings with Argentine Spanish data, aiming to implement prominence detection in an ASR system, and further investigating how specific acoustic properties of broad focus utterances influence listener decisions. The researchers also plan to explore the practical issue of how to best integrate prominence information into ASR systems to enhance word hypothesis generation.

Summary updated

BibTeX
@inproceedings{mixdorff2015acoustic,
  author = {Hansjörg Mixdorff and Christian Cossio-Mercado and Angelika Hönemann and Jorge Gurlekian and Diego Evin and Humberto Torres},
  title = {{Acoustic correlates of perceived syllable prominence in German}},
  booktitle = {Interspeech},
  year = {2015},
}
Figure