Acoustic Analysis of Syllables Across Indian Languages

Anusha Prakash, Jeena J. Prakash, Hema A. MurthyInterspeech 2016

TL;DR

This paper analyzes the acoustic properties of syllables across six Indian languages to identify similarities and differences, aiming to improve multilingual speech technology development. By comparing syllable structures and acoustic features, the researchers found that Indo-Aryan and Dravidian languages exhibit distinct patterns, which can be leveraged for more efficient text-to-speech (TTS) synthesis through cross-language model borrowing.

What Problem Does This Paper Address?

Developing robust speech technologies (like speech recognition and synthesis) for the vast linguistic landscape of India, with over 1652 languages and widespread bilingualism, is a significant challenge. Existing approaches often focus on individual languages, which is inefficient and costly. This paper addresses the need for a more unified approach by investigating the fundamental acoustic properties of syllables, the basic units of speech production, across different Indian language families (Indo-Aryan and Dravidian). The goal is to leverage shared linguistic characteristics to build better multilingual speech systems, particularly for text-to-speech (TTS) synthesis, which often relies on syllable-level segmentation.

Methods

The core of this research involves an acoustic analysis of syllables extracted from continuous speech across six Indian languages: Bengali, Hindi, Marathi (Indo-Aryan), and Kannada, Tamil, Telugu (Dravidian).

  1. Syllable Segmentation: A novel hybrid segmentation algorithm is employed. This algorithm combines machine learning (HMMs) with signal processing techniques (group-delay, short-term energy, spectral flux) to accurately identify syllable boundaries in continuous speech. This is an improvement over traditional methods that might not consider boundary information effectively.
  2. Feature Extraction: For the most frequent syllables (top 300), acoustic features such as duration, average energy, and average fundamental frequency (f0) are extracted. These features are normalized to account for speaker-specific variations.
  3. Cross-Language Comparison: Syllables are mapped to a common label set to enable direct comparison across languages, regardless of their native scripts. The analysis focuses on syllable structure (number of phones) and word structure (number of syllables per word).
  4. TTS System Development: The insights gained from the acoustic analysis are applied to build statistical parametric TTS systems. A key technique used is cross-borrowing of Hidden Markov Models (HMMs). Instead of training HMMs from scratch for each language, models from a similar language (e.g., Hindi for other Indo-Aryan languages, Tamil for Dravidian) are used as initial models for segmentation and training.

Datasets

The study utilizes speech data from six Indian languages:

  • Indo-Aryan: Bengali, Hindi, Marathi
  • Dravidian: Kannada, Tamil, Telugu

Both male and female speech data were used for these languages. Additionally, 5 hours of Rajasthani (Indo-Aryan) male and female data were used to further validate the acoustic analysis. The paper does not explicitly mention the source or size of these datasets beyond the duration (e.g., 3.56 to 5.37 hours per language/gender pair).

Key Insights and Ideas

  • Syllable Structure Variation: Syllables primarily consist of two phones (CV structure) or three phones (CVC structure) across all studied languages. Syllables with one phone (vowels) and four or more phones are less common, with the latter often arising from loanwords (e.g., English).
  • Word Structure Differences: Indo-Aryan languages tend to have more bi-syllabic and tri-syllabic words, while Dravidian languages commonly feature words with three or more syllables. This difference is attributed to the agglutinative nature of Dravidian languages, where multiple morphemes are combined into single words.
  • Duration Patterns: Male speakers of Dravidian languages tend to have shorter average syllable durations compared to Indo-Aryan languages. This pattern was less pronounced in female speakers, suggesting potential gender-specific prosodic variations.
  • Cross-Language Model Borrowing Efficacy: Using HMMs from a source language to build TTS systems for a target language significantly improves synthesis quality (measured by DMOS and WER) compared to using HMMs from a different language family. This highlights the effectiveness of leveraging similarities between related languages.

Key Results

  • Improved TTS Quality: TTS systems built using HMMs borrowed from a similar language (e.g., Hindi for Bengali, Tamil for Telugu) achieved better synthesis quality (lower DMOS and WER) than systems built with HMMs from unrelated languages. For instance, Bengali TTS using Hindi HMMs yielded a DMOS of 2.99 and WER of 9.82, while using Tamil HMMs resulted in a DMOS of 3.24 and WER of 3.67.
  • Acoustic Similarities: The analysis revealed distinct acoustic patterns between Indo-Aryan and Dravidian language groups, particularly in syllable duration.
  • Syllable Structure Dominance: Across all languages, CV and CVC syllable structures are dominant, forming the vast majority of common syllables.

Limitations and Future Work

  • Gender Anomaly: The paper notes an anomaly where the syllable duration distributions for female data in Hindi and Bengali did not conform to the expected Indo-Aryan pattern and overlapped with Dravidian languages. Further studies are suggested to investigate this.
  • Limited Acoustic Features: While duration, energy, and f0 were analyzed, the paper mentions that conclusive results were not obtained for energy and f0, and these were not presented. Future work could delve deeper into these features.
  • Cross-Gender Analysis: The current study focused on language-specific perspectives rather than cross-gender analysis. Future research could explore gender-based prosodic differences more thoroughly.
  • Broader Language Coverage: While six languages were studied, the authors suggest that further studies across more Indian languages could provide a more comprehensive understanding of linguistic similarities and differences.
  • Low-Resource Languages: The findings are particularly relevant for developing speech technologies for low-resource Indian languages, where data is scarce. The cross-borrowing technique offers a promising avenue for such scenarios.

Summary updated

BibTeX
@inproceedings{prakash2016acoustic,
  author = {Anusha Prakash and Jeena J. Prakash and Hema A. Murthy},
  title = {{Acoustic Analysis of Syllables Across Indian Languages}},
  booktitle = {Interspeech},
  year = {2016},
}
Figure