Unsupervised learning of vowels from continuous speech based on self-organized phoneme acquisition model

Kouki Miyazawa, Hideaki Kikuchi, Reiko MazukaInterspeech 2010

TL;DR

This paper proposes a novel unsupervised learning model, the Self-Organized Phoneme Acquisition Model (SOPAM), that mimics how infants learn vowel sounds from continuous, natural speech without explicit labels. The model successfully learns Japanese vowel categories with accuracy comparable to human adults, suggesting that natural speech contains sufficient information for phoneme acquisition.

What Problem Does This Paper Address?

The paper tackles the fundamental question of how humans, particularly infants, acquire their native phoneme systems without explicit instruction. Current speech processing technologies rely on supervised learning with vast amounts of labeled data, which is costly and inflexible. While previous computational models have explored unsupervised phoneme learning, they often use read speech or simplified acoustic features, failing to replicate the complexity of natural, continuous speech environments infants experience. This study aims to bridge this gap by developing a model that learns vowel categories from natural, continuous Japanese speech, mimicking the human ability to extract relevant acoustic information.

Methods

The core of the proposed method is a Self-Organizing Map (SOM), a type of unsupervised neural network known for its ability to classify high-dimensional data and organize it spatially. The authors adapt the SOM for phoneme acquisition through several key steps:

  • Spectral Time Series Analysis: Speech is preprocessed to extract acoustic features mimicking the human auditory system. This involves calculating Mel Frequency Cepstral Coefficients (MFCCs) to capture spectral envelope information and AMFCC (Acoustic MFCCs) to represent dynamic spectral changes.
  • Unsupervised Clustering with SOM: The preprocessed speech features are fed into a one-dimensional SOM. The SOM learns to cluster these features by adjusting its internal "reference vectors" to match the input data. The learning process involves an "ordering phase" with high learning rates and a "tuning phase" with lower rates to refine the clusters.
  • Unification of Categories: A crucial innovation is a method to unify similar categories learned by the SOM. This is achieved by constructing a histogram based on the density of data points assigned to each SOM node and the similarity between neighboring nodes. This process allows the model to estimate the number and boundaries of vowel categories without prior knowledge, mirroring how infants might consolidate phonetic distinctions.

Datasets

  • Corpus of Spontaneous Japanese (CSJ): This dataset was used for both training and evaluation. The authors extracted 100 seconds of continuous speech for evaluation and 300 seconds for training. The CSJ dataset provides phoneme-level time information, which was used to extract specific vowel sounds for evaluation.
  • Details of Speech Used: The paper details the speakers (12 in total, varying sex and speech style: Academic Presentation Speech (APS), Simulated Public Speech (SPS), and re-reading speeches) and the audio specifications (16-bit, 16 KHz).

Key Insights and Ideas

  • Natural Speech Sufficiency: The study demonstrates that natural, continuous speech, even in relatively short durations (100 seconds), contains sufficient information for unsupervised learning of vowel categories. This challenges the notion that infants require highly structured or simplified input.
  • SOM's Suitability for Vowel Learning: The SOM's ability to learn from unlabeled data and its spatial organization make it a suitable model for capturing the stationary acoustic characteristics of vowels.
  • Category Unification as a Cognitive Parallel: The proposed method for unifying categories based on data density and similarity offers a plausible computational parallel to how infants might consolidate phonetic distinctions into native phonemes.
  • Robustness to Speech Style and Sex: The model's performance showed no significant difference across different speech styles (APS, SPS, re-reading) or between male and female speakers, suggesting a degree of robustness.

Key Results

  • Vowel Recognition Accuracy: The model achieved an average vowel recognition accuracy of 63.5%. This accuracy was comparable to human performance reported in previous studies (Table 3 shows human accuracy around 58%).
  • Speaker and Vowel Variation: Accuracy varied between speakers (47.3% to 75.9%) and vowels, with /e/ being the most accurately recognized and /u/ the least.
  • Number of Clusters: On average, the SOM model obtained 10.6 clusters, indicating its ability to segment the acoustic space into distinct categories.
  • Training Data Quantity: The vowel identification rate did not significantly change when the training data quantity was reduced to 100 seconds, suggesting the model reaches a convergence state relatively quickly.

Limitations and Future Work

  • Consonant Learning: The paper acknowledges that the model's performance on consonants is not discussed, and consonants have different acoustic characteristics (unsteady short-time spectrum) that may require different modeling approaches.
  • Infant-Directed Speech (IDS): The authors plan to investigate the role of Infant-Directed Speech (IDS), which is known to differ acoustically from adult-directed speech, in phoneme acquisition.
  • Further Analysis of Acoustic Features: While MFCCs and AMFCCs were used, future work might explore other acoustic features or combinations thereof.
  • Model Complexity: The paper mentions that the model could not categorize consonants, suggesting that the current model might be more suited for vowels. Future work will focus on extending the model to handle consonants.

Summary updated

BibTeX
@inproceedings{miyazawa2010unsupervised,
  author = {Kouki Miyazawa and Hideaki Kikuchi and Reiko Mazuka},
  title = {{Unsupervised learning of vowels from continuous speech based on self-organized phoneme acquisition model}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure