A computational model for unsupervised word discovery

Louis ten Bosch, Bert CranenInterspeech 2007

TL;DR

This paper introduces a novel computational model for discovering words and word-like speech fragments directly from raw audio, without relying on pre-defined lexicons or acoustic phone models. This approach holds promise for both advancing speech recognition by freeing it from traditional constraints and for modeling how infants learn language by extracting emergent word properties from continuous speech.

What Problem Does This Paper Address?

Traditional Automatic Speech Recognition (ASR) systems are heavily reliant on pre-defined lexicons and acoustic phone models. This creates a significant challenge when encountering "out-of-vocabulary" (OOV) words, as the system simply cannot recognize them. Furthermore, this approach doesn't reflect how humans, particularly infants, learn language. Babies can segment continuous speech into words and learn their meanings without explicit instruction or pre-existing linguistic knowledge. This paper aims to bridge this gap by developing an unsupervised method for word discovery that mimics this natural learning process and offers a more flexible alternative for ASR.

Methods

The proposed system, termed Statistical Word Discovery (SWD), operates in three stages:

  1. Automatic Segmentation: This stage extracts acoustic features (MFCCs, log energy, and their derivatives) from the speech signal. It then uses a data-driven approach to identify potential segment boundaries within utterances. This is achieved by analyzing the similarity between acoustic frames and identifying local maxima in a distance function, weighted by signal energy to avoid flagging silent portions. The output is a sequence of hypothesized speech segments for each utterance.
  2. Labelling of Segments: The segments identified in Stage 1 are then clustered using a k-means algorithm. The distance between segments is calculated using Dynamic Time Warping (DTW) on their acoustic feature vectors. Each cluster is assigned an abstract label (a unique integer), and each segment inherits the label of its cluster. This stage transforms the raw acoustic segments into symbolic representations.
  3. Word Discovery by DTW: This is the core word discovery stage. It takes the sequence of abstract labels from Stage 2, along with abstract "tags" indicating the presence of specific words in an utterance (provided as input, mimicking a high-level semantic cue). The algorithm compares utterances by performing DTW on their label sequences. It identifies matching subsequences and, crucially, uses the abstract tags to differentiate between hypothesized word occurrences (put into a "Bmatch" set) and non-word occurrences (put into a "Bno-match" set). By iteratively processing utterances and comparing these sets, the algorithm identifies recurring label sequences that are likely to represent words.

Datasets

The primary dataset used is the Aurora 2.0 database, specifically the clean part containing utterances from male speakers. This database consists of digit strings (e.g., "one eight one seven four") and was chosen for its low perplexity and availability of multiple copies with varying signal-to-noise ratios, making it suitable for testing and future studies in noisy conditions. The paper notes that the clean subset contains 4220 utterances, with an average of 3.2 digits per utterance.

Key Insights and Ideas

  • Unsupervised Bootstrapping: The model demonstrates the ability to learn word-like units from raw speech data without any prior linguistic knowledge, mirroring aspects of infant language acquisition.
  • Hierarchical Representation: The three-stage process creates a hierarchy, moving from low-level acoustic features to symbolic segment labels, and finally to hypothesized word sequences.
  • Leveraging Abstract Cues: The use of abstract "tags" in Stage 3 is crucial. It acts as a high-level semantic hint, allowing the system to distinguish between actual word occurrences and spurious matches, significantly improving the discovery process.
  • DTW as a Unifying Tool: Dynamic Time Warping is employed in both segment labeling (Stage 2) and word discovery (Stage 3), providing a consistent mechanism for comparing temporal sequences.

Key Results

The paper reports promising performance on the Aurora 2.0 digit-string database. At 1000 observed word tokens per digit, the algorithm achieves:

  • Word Accuracy: 90%
  • False Alarms: Decreasing with more data.
  • False Rejections: Decreasing with more data.

The authors note that the number of false alarms is relatively high, suggesting potential for further refinement in defining "optimal" symbolic subsequences.

Limitations and Future Work

  • Reliance on Abstract Tags: The current model requires abstract tags to indicate word presence. Future work aims to explore how to learn these tags or discover words without them, moving closer to true unsupervised learning.
  • Performance in Noisy Conditions: While the Aurora 2.0 database offers varying SNRs, the current study focuses on the clean subset. Extending the algorithm to perform robustly in noisy environments is a key future direction.
  • Refining Subsequence Optimality: The authors suggest that the precise definition of optimality for symbolic subsequences in Stage 3 could be improved to reduce false alarms.
  • Exploration of Paralinguistic Information: Future research will investigate the use of prosodic features (pauses, energy, pitch) to further facilitate word detection.
  • Modeling Language Acquisition: The authors are interested in further developing this model as a computational account of language acquisition, particularly the interplay between phone and word learning.

Summary updated

BibTeX
@inproceedings{bosch2007computational,
  author = {Louis ten Bosch and Bert Cranen},
  title = {{A computational model for unsupervised word discovery}},
  booktitle = {Interspeech},
  year = {2007},
}
Figure