Perception-based automatic approximation of F0 contours in Cantonese speech

Yujia Li, Tan LeeInterspeech 2010 · DOI

Abstract

In tonal languages like Cantonese, the fundamental frequency (F0) contour is the primary carrier of linguistic meaning. However, raw F0 data is often cluttered with micro-prosody and non-perceivable variations. This paper proposes a three-step automatic framework to simplify these complex contours into linear segments based on human auditory perception. The system achieves a 93.1% success rate in producing speech judged as natural as the original recordings, demonstrating that perceptually-driven linear approximation is a viable alternative to manual processing for large-scale prosody research.

Introduction

The fundamental frequency (F0), or pitch, is critical for meaning in Cantonese. Raw F0 contours recorded from human speech contain "micro-prosody"—tiny fluctuations caused by the physical constraints of the vocal tract—and production-intended variations that are often imperceptible to listeners. While manual linear approximation can strip away this irrelevant data without losing naturalness, the process is slow and inconsistent. This study introduces an automated pipeline to standardize this simplification process while preserving the perceptual "soul" of the speech.

Methodology

The proposed automatic pipeline consists of three primary stages:

  1. Contour Smoothing: Moving-window averaging is applied to the raw F0 data to flatten minor fluctuations and remove outliers, highlighting the significant trends in pitch change.
  2. Locating Turning Points: The algorithm identifies global peaks and valleys using a "Global Search" method. It defines reference lines and selects candidates based on distance and time constraints, which are then refined according to syllable count and tone types.
  3. F0 Value Determination (WTA Model): Instead of using raw pitch values at turning points, the system employs the Weighted Time Average (WTA) model. This model predicts human pitch perception for a segment. By solving linear equations derived from the WTA model, the system ensures that the simplified linear segment matches the perceived start and end pitch of the original complex curve.
Linear approximation of F0 contours showing the distinction between tone and transition segments.

The framework specifically distinguishes between "tone contours" (the linguistically meaningful portions) and "transition contours" (the movements between tones), prioritizing the accuracy of the former due to higher human sensitivity.

Experimental Results

The framework was evaluated using a database of 335 Cantonese polysyllabic word utterances from a native female speaker. The evaluation focused on level tones to establish a performance baseline.

Comparison of manually marked turning points versus those automatically located by the proposed algorithm.

Key performance metrics include:

  • Perceptual Naturalness: In listening tests with 20 subjects, 93.1% of automatically approximated utterances were judged comparable to natural speech, nearly matching the 97.3% achieved by manual approximation.
  • Turning Point Accuracy: The algorithm matched manually identified turning points with an 87.5% success rate (within a 35-45ms tolerance).
  • Slope Consistency: The average slope ratio between automatic and manual approximations was 1.4, significantly below the perceptual threshold of 2.0 required for humans to detect differences in pitch movement.

Discussion and Conclusion

The core insight of this work is that approximation does not require perfect geometric fidelity to the original signal; it only needs to satisfy the constraints of human auditory perception. By leveraging psychoacoustic limits—such as the "Glissando Threshold"—the system effectively simplifies speech prosody.

While currently optimized for level tones and a single speaker, the underlying smoothing and F0 determination steps are largely language-independent. Future work will focus on extending the turning point logic to accommodate more complex "rising tones" and validating the model across a more diverse range of speakers and languages.

Summary updated

BibTeX
@inproceedings{li2010perception,
  author = {Yujia Li and Tan Lee},
  title = {{Perception-based automatic approximation of F0 contours in Cantonese speech}},
  booktitle = {Interspeech},
  year = {2010},
  doi = {10.21437/interspeech.2010-432},
}
Figure