Text-based unstressed syllable prediction in Mandarin

Ya Li, Jianhua Tao, Meng Zhang, Shifeng Pan, Xiaoying XuInterspeech 2010 · OpenReview

Abstract

Mandarin word stress is a critical component for enhancing the naturalness of speech synthesis. This research focuses on the prediction of unstressed syllables, which significantly impact the intelligibility and prosody of synthetic speech. Detecting stress from text is challenging due to complex contextual dependencies. We propose a method using a Classification and Regression Tree (CART) model to predict unstressed syllables with 85% accuracy. Integrated into a Text-to-Speech (TTS) system, this approach improved the Mean Opinion Score (MOS) by 0.35 and resulted in pitch contours that more closely align with natural speech.

Introduction

Achieving human-like naturalness in Mandarin Text-to-Speech (TTS) systems remains a significant challenge. While Mandarin is primarily known as a tonal language, it also employs stress levels—stressed, regular, and unstressed—to convey meaning and rhythmic structure. Unstressed syllables are particularly complex; they are characterized not just by reduced volume, but by a distinct drop in pitch, a narrower pitch range, and shorter duration.

The semantic importance of stress is evident in minimal pairs: for example, dong1 xi1 (east and west) versus dong1 xi0 (thing). Predicting these syllables from raw text is difficult because their occurrence is governed by intricate syntactic and contextual rules. This paper aims to bridge this gap by identifying unstressed syllables using textual features to improve acoustic unit selection in TTS systems.

Methodology

The researchers framed the identification of unstressed syllables as a "one-versus-others" classification task.

The CART Model

A Classification and Regression Tree (CART) was employed to make decisions based on a hierarchy of linguistic features. The model utilizes a sliding window approach, considering the target syllable along with its immediate neighbors.

Feature Engineering

Five primary linguistic features were extracted from the text to serve as inputs for the model:

  1. Initials and Finals: The phonetic components of the syllable.
  2. Tone: The four standard tones plus the neutral tone.
  3. Part-of-Speech (POS): Utilizing the PKU POS tag set (44 classes).
  4. Position in Word: Identifying if a syllable is at the start, middle, or end of a word, or is a standalone monosyllabic word.
Feature template and class distribution used for the CART model.

Dataset and Integration

The model was trained on a proprietary corpus of 6,000 sentences recorded by a professional female speaker. Three experts manually labeled the stress levels. The dataset contained 4,688 unstressed syllables (approximately 4% of the total). The prediction model was integrated into the Wiston TTS system, a corpus-based concatenative synthesizer.

Results and Discussion

The CART model achieved an overall prediction accuracy of approximately 85%. Analysis revealed that Part-of-Speech (POS), word position, and the syllable's initial are the most significant predictors of stress.

Subjective and Objective Evaluation

Subjective listening tests involving seven speech experts showed a significant improvement in the Mean Opinion Score (MOS), which rose from 3.86 to 4.21. Objectively, the average distance between synthesized pitch contours and natural speech decreased from 32 Hz to 28 Hz.

Comparison of pitch contours for natural speech, the new Wiston TTS, and the original system. The new system shows a closer match to natural speech for the unstressed syllable 'ji0'.

The model demonstrated higher precision (86.3%) than recall (56.3%). This indicates that while the system reliably identifies grammatically required unstressed syllables, it struggles with "provisional" stress—instances influenced by speaker mood or specific intonation that are not easily captured by shallow text analysis.

Conclusion

This study demonstrates that text-based prediction of unstressed syllables significantly enhances the naturalness of Mandarin speech synthesis. While the current model relies on shallow morphological features, future work will explore deeper semantic and emotional contexts to improve recall. Additionally, the researchers plan to transition to HMM-based synthesis to better address data sparseness and expand the model to a full three-class stress system.

Summary updated

Abstract

Recently, an increasing attention has been paid to Mandarin word stress which is important for improving the naturalness of speech synthesis. Most of the research on Mandarin speech synthesis focuses on three stress levels: stressed, regular and unstressed. This paper emphasizes the unstressed syllable prediction because the unstressed syllable is also important to the intelligibility of the synthetic speech. Similar as the prosodic structure, it is not easy to detect the stress from text analysis due to the complicated context information. A method based on Classification and Regression Tree (CART) model has been proposed to predict the unstressed syllables with the high accuracy of 85%. The method has been finally applied into the TTS system. The experiment shows that the MOS score of synthetic speech has been improved by 0.35; the pitch contour of the new synthesized speech is also closer to natural speech.
BibTeX
@inproceedings{li2010text,
  author = {Ya Li and Jianhua Tao and Meng Zhang and Shifeng Pan and Xiaoying Xu},
  title = {{Text-based unstressed syllable prediction in Mandarin}},
  booktitle = {Interspeech},
  year = {2010},
  url = {https://openreview.net/forum?id=E0CFXc4Diu},
}
Figure