Two new estimation methods for a superpositional intonation model

Humberto M. Torres, Hansjörg Mixdorff, Jorge A. Gurlekian, Hartmut R. PfitzingerInterspeech 2010

TL;DR

This paper introduces two novel methods for automatically estimating the parameters of the Fujisaki intonation model, a popular tool for modeling speech prosody. By incorporating linguistic cues like accent types and phrase boundaries, these new methods significantly outperform existing automatic approaches and even rival manual annotation, paving the way for more accurate and natural text-to-speech systems.

What Problem Does This Paper Address?

Modeling human intonation is crucial for creating natural-sounding synthetic speech. The Fujisaki model, a widely used framework, describes intonation as a superposition of a base frequency, tonal accents, and phrase accents. However, a major hurdle has been the accurate and automatic estimation of the model's parameters directly from speech data. Existing automatic methods, while convenient, often produce spurious or linguistically nonsensical accents, requiring manual correction. This paper aims to bridge this gap by developing new automatic estimation methods that leverage linguistic knowledge to improve accuracy and reduce reliance on manual intervention.

Methods

The authors build upon Mixdorff's existing automatic parameter estimation method, which serves as their baseline. Their core innovation lies in introducing two new estimation methods that incorporate linguistic hypotheses to initialize the parameter extraction process.

  • Linguistic Initialization: Instead of relying solely on acoustic cues, the new methods use linguistic information to guide the initial parameter settings. This includes assumptions about the placement of accent commands relative to linguistic accents (tonal or lexical) and phrase boundaries.
  • Two New Methods:
    • Tonal Accent Method: This method uses tonal accents (movements in the intonation contour that mark contrasts) to inform the placement of accent commands.
  • Lexical Accent Method: This method utilizes lexical accents (intrinsic properties of words) to guide accent command placement.
  • Relaxed Constraints: The authors also removed some of the original method's restrictions on parameter values (e.g., minimum/maximum amplitudes, durations, and distances between commands), allowing for more flexibility.
  • Analysis-by-Synthesis: Both new methods, like the baseline, employ an iterative analysis-by-synthesis procedure to refine the estimated parameters by minimizing the error between the synthesized and original intonation contours.

Datasets

The experiments were conducted using the IMS Radio News Corpus, which consists of German news texts read by professional speakers. The reference data for the Fujisaki model was extracted automatically and then manually corrected. The authors selected 73 news articles read by one male speaker, totaling 48 minutes of speech, with 1,670 seconds of voiced segments.

Key Insights and Ideas

  • Linguistic Cues are Powerful: Incorporating linguistic information, specifically the types and locations of accents and phrase boundaries, significantly improves the accuracy of automatic intonation parameter estimation.
  • Tonal vs. Lexical Accents: Both tonal and lexical accent information can be effectively used to guide parameter estimation, with the tonal accent method showing slightly better performance in their experiments.
  • Reduced Manual Effort: The proposed methods offer a substantial improvement over existing automatic methods, reducing the need for laborious manual correction, which is a major bottleneck in current workflows.
  • Model Flexibility: Removing overly strict constraints on parameter values allows the model to better adapt to the nuances of real speech, as demonstrated by the improved performance.

Key Results

  • Superior Performance: The new linguistically motivated methods significantly outperform Mixdorff's automatic method.
  • Competitive with Manual: The proposed methods achieve results that are slightly better than the manually corrected reference method, demonstrating their high accuracy.
  • Quantitative Improvements: While specific RMSE values vary across experiments, the paper consistently shows lower RMSE (Root Mean Square Error) for the new methods compared to the automatic baseline. For instance, in the tonal method, RMSE was around 1.42 ST (semitones) for the automatic initialization versus 0.49 ST for the tonal prototype initialization, and the manual method achieved 1.48 ST.
  • Accent Density: The new methods maintain a higher accent density compared to some reference methods, suggesting they are capturing more relevant prosodic events.

Limitations and Future Work

  • Data Dependency: The lexical accent method's results were limited by the availability of labeled data for lexical accents.
  • Physiological Studies: The authors suggest that further physiological studies are needed to determine the actual physiological constraints on the Fujisaki model parameters.
  • Cross-Lingual Adaptation: An important future direction is to adapt and test these hypotheses for languages other than German.
  • Automatic Tonal Tagger: The demonstrated link between lexical accents and accent commands could be leveraged to build a more effective automatic tonal tagger.

Summary updated

BibTeX
@inproceedings{torres2010two,
  author = {Humberto M. Torres and Hansjörg Mixdorff and Jorge A. Gurlekian and Hartmut R. Pfitzinger},
  title = {{Two new estimation methods for a superpositional intonation model}},
  booktitle = {Interspeech},
  year = {2010},
}

Loading related content...

Figure