Unit selection based speech synthesis for poor channel condition

Ling Cen, Minghui Dong, Paul Y. Chan, Haizhou Li

Abstract

This paper introduces a specialized Text-to-Speech (TTS) system designed to maintain high intelligibility under poor channel conditions, such as noisy telephone lines. The system incorporates speech intelligibility metrics directly into the unit selection process, mimics the human Lombard effect, and applies advanced channel equalization. Experimental results show that these modifications significantly improve clarity and listener preference in challenging acoustic environments.

Introduction

Most speech synthesis research focuses on maximizing naturalness under ideal listening conditions. However, synthetic voices transmitted over "poor channels"—characterized by low bandwidth or heavy background noise—often become unintelligible. Standard TTS systems typically do not account for how noise masks specific speech components. To address this, the authors propose a system that mimics human adaptation to noise, where speakers adjust rhythm and emphasize consonants rather than just increasing overall volume.

Proposed Methodology

The architecture consists of three primary components designed to combat channel-induced degradation.

Figure 1: Flowchart of the proposed unit selection based TTS system architecture.

1. Intelligibility-Aware Unit Selection

In traditional unit selection, units are chosen based on target text matching and concatenation smoothness. This system introduces an "intelligibility cost" using the Speech Intelligibility Index (SII). By predicting how well a unit will survive expected channel noise, the system prioritizes naturally robust units.

2. Lombard-Inspired Prosody Modification

The system simulates the Lombard effect by:

  • Boosting Unvoiced Phonemes: Increasing the amplitude of segments like 's', 't', and 'p', which are highly susceptible to noise masking.
  • Temporal Lengthening: Increasing speech duration by a factor of 1.2x to provide listeners more processing time.

3. Channel Equalization

A Finite Impulse Response (FIR) filter is designed using convex optimization (specifically Second Order Cone Programming). This equalizer minimizes the discrepancy between the actual channel response and a desired clean response, mitigating linear distortions.

Experimental Setup and Results

The system was trained on the British English corpus from the Blizzard Challenge 2009, comprising 15 hours of speech across diverse domains. Evaluation involved subjective listening tests with 10 participants and 40 test sentences.

Intelligibility and Preference

The proposed system was preferred over a baseline TTS in 68% of head-to-head comparisons. Quantitative analysis using SII confirmed that targeted amplification of unvoiced segments directly correlates with improved intelligibility.

Figure 6: Relationship between amplitude gain of unvoiced segments and the Speech Intelligibility Index (SII).

As shown in Figure 6, increasing the amplitude gain from 1.1 to 2.0 leads to a consistent rise in the SII statistic, validating the effectiveness of the Lombard-inspired modifications.

Conclusion

By integrating environmental awareness into the synthesis pipeline, the proposed system significantly enhances speech robustness in poor channel conditions. While the 1.2x lengthening factor provides a balanced improvement, future work will focus on phoneme-specific prosody modifications and reducing the computational overhead of real-time SII calculations and convex optimization.

Summary updated

BibTeX
@inproceedings{cen2009unit,
  author = {Ling Cen and Minghui Dong and Paul Y. Chan and Haizhou Li},
  title = {{Unit selection based speech synthesis for poor channel condition}},
  booktitle = {Interspeech},
  year = {2009},
  doi = {10.21437/interspeech.2009-595},
}
Figure