A genetic algorithm for unit selection based speech synthesis

Rohit KumarInterspeech 2004

TL;DR

This paper introduces a Genetic Algorithm (GA) approach to the unit selection problem in concatenative speech synthesis. By treating unit selection as a global optimization problem, the GA aims to find more natural-sounding speech by exploring a wider range of possibilities than traditional local optimization methods.

What Problem Does This Paper Address?

Modern text-to-speech (TTS) systems often rely on concatenative synthesis, where pre-recorded speech units (like syllables or phones) are stitched together to form spoken sentences. The core challenge lies in selecting the best instance of each required unit from a large database, considering both how well each unit matches the target speech characteristics (target cost) and how smoothly adjacent units connect (join cost). With millions of possible combinations for even moderately long sentences, exhaustively searching for the optimal sequence is computationally infeasible. Traditional methods often use local optimization (like hill climbing), which can get stuck in suboptimal solutions. This paper addresses the need for a more robust global optimization strategy to improve speech naturalness.

Methods

The authors propose using a Genetic Algorithm (GA) to tackle the unit selection problem. The GA operates on a population of potential solutions (sequences of unit instances). Key components of their GA include:

  • Representation: Each "individual" in the population is a sequence of selected unit instances corresponding to the target utterance.
  • Initialization: The initial population is created by distributing unit instances across individuals to cover a broad search space.
  • Fitness Function (Unfitness): The "unfitness" of an individual is calculated based on a weighted sum of prosodic (pitch, duration, energy) and linguistic (phonetic context, syllable nuclei) mismatches between adjacent units. Lower unfitness (or higher fitness) indicates a better solution.
  • Selection: Individuals with better fitness have a higher probability of being selected for the next generation.
  • Crossover: Two parent individuals are combined to create offspring. The authors describe a specific crossover method that intelligently chooses unit instances for a shared segment of the sequence based on minimizing mismatch.
  • Elitism: The fittest individuals from the current generation are directly carried over to the next, ensuring that good solutions are not lost.
  • No Mutation: Notably, the authors state they do not use mutation in their implementation.
  • Termination Criteria: Evolution stops when the improvement in the best individual's fitness plateaus or after a maximum number of generations (50).

Datasets

The paper uses a Hindi speech database. This database consists of basic units at syllable and phone levels, with each instance stored along with prosodic and linguistic features. Instances are pre-sorted by a "global prosodic mismatch function" (GPMF) value. No new benchmarks are introduced; the evaluation is based on synthesized sentences.

Key Insights and Ideas

  • Global vs. Local Optimization: The paper highlights the inherent trade-off between the computational cost of global optimization and the risk of suboptimal solutions with local optimization in the context of unit selection.
  • GA's Potential for Naturalness: By exploring a wider search space, GAs offer a promising avenue for achieving more natural-sounding speech synthesis than methods prone to local optima.
  • Importance of Mismatch Functions: The effectiveness of the GA hinges on well-defined mismatch functions that accurately capture the perceptual quality of unit selection and concatenation. The paper details their specific weighted prosodic and linguistic mismatch calculations.
  • Parameter Sensitivity: The authors experiment with population size and crossover probability, suggesting that a higher crossover probability (around 0.8) and a sufficiently large population are beneficial, though computational cost is a consideration.

Key Results

  • Quantitative Comparison: In a perceptual evaluation with 8 native Hindi speakers, the GA-based synthesizer achieved an average score of 3.013 with a variance of 0.5375. This was compared to a local optimization-based synthesizer which scored 3.125 with a variance of 1.1392.
  • Qualitative Findings: While the local optimization method received a slightly higher average score, the GA-based synthesizer demonstrated significantly lower variance, indicating more consistent and predictable output quality.
  • Convergence: Figures in the paper show that the GA population converges towards better solutions over generations, with the variance between individuals decreasing.

Limitations and Future Work

  • No Mutation: The absence of mutation is a limitation, as mutation is typically a crucial operator for exploring novel solutions and preventing premature convergence.
  • Initial Population Creation: The authors acknowledge that there is scope for improvement in how the initial population is created to better cover the search space.
  • Computational Cost: While GAs can be more efficient as databases grow, they still require significant computation, and the paper notes that higher population sizes increase computation time.
  • Future Work: The authors suggest further research into improving the initial population generation strategy and exploring the impact of mutation. They also believe GA-based unit selection can offer faster results as unit selection databases expand.

Summary updated

BibTeX
@inproceedings{kumar2004genetic,
  author = {Rohit Kumar},
  title = {{A genetic algorithm for unit selection based speech synthesis}},
  booktitle = {Interspeech},
  year = {2004},
}
Figure