A multi-class approach for modelling out-of-vocabulary words

Issam Bazzi, James R. GlassInterspeech 2002 · DOI

TL;DR

This paper tackles the challenge of out-of-vocabulary (OOV) words in speech recognition by proposing a "multi-class" approach. Instead of a single generic model for all unknown words, they create multiple specialized OOV models based on word categories (like part-of-speech or phonetic similarity). This significantly reduces false alarms and improves recognition accuracy, especially in domains with many OOV words.

What Problem Does This Paper Address?

Continuous speech recognition (CSR) systems rely on a predefined vocabulary. When a spoken word falls outside this vocabulary, it's an "out-of-vocabulary" (OOV) word. OOV words are a major source of errors in CSR systems, leading to a significant increase in Word Error Rate (WER). Traditional approaches often try to handle all OOV words with a single, generic model, which can be inefficient and lead to many false alarms (mistaking an in-vocabulary word for an OOV word or vice-versa). This paper aims to improve OOV detection and reduce false alarms by creating more specialized OOV models.

Methods

The core innovation is extending a previous OOV modeling approach from a single OOV model to multiple OOV models, each representing a "class" of OOV words. The system's search network is augmented with these multiple OOV models, allowing the recognizer to choose the most appropriate OOV model for a given word.

Two primary methods for defining these OOV classes are explored:

  1. Part-of-Speech (POS) Classes: Words are categorized based on their grammatical function (e.g., noun, verb, adjective). This leverages existing linguistic information. To handle words that can belong to multiple POS categories (like "book" being a noun or verb), "intersection classes" are created.
  2. Automatically-Derived Classes (Clustering): This approach uses a two-step clustering procedure:
  3. Step 1: Agglomerative Clustering: Words are grouped based on their phonetic similarity using a phone-pair edit distance. This is a bottom-up hierarchical clustering method. Due to computational cost, this is initially performed on a subset of words.
  4. Step 2: Perplexity Clustering: This is an iterative refinement step, similar to K-means. Words are moved between classes if doing so reduces the overall model perplexity (a measure of how well a probability model predicts a sample). This step can be initialized either by the agglomerative clustering results or by POS tags.

The paper also explores different ways to integrate these multi-class models into the recognition system, including using multiple OOV networks and/or multiple OOV n-gram language models.

Datasets

  • PRONLEX Dictionary: Used for its large vocabulary (90,694 words) and multiple pronunciations. This dictionary is the source for identifying OOV words and their phonetic representations.
  • COMLEX Dictionary: Used to obtain POS tags for words in PRONLEX.
  • JUPITER Weather Information Domain: This domain is used for training and testing the speech recognition system. The training set consists of 88,755 utterances, and the test set has 2,029 utterances, with 314 containing OOV words.

Key Insights and Ideas

  • Specialization is Key: The fundamental idea that treating OOV words as a monolithic group is suboptimal. Creating specialized models for different types of OOV words leads to significant performance gains.
  • POS Tags as a Good Starting Point: Using readily available POS tags provides a surprisingly effective way to define OOV classes, offering a good balance between performance and computational cost.
  • Clustering for Fine-Tuning: While POS tags are good, data-driven clustering (especially when initialized with POS tags) can further refine class assignments and reduce perplexity, leading to even better results.
  • OOV Networks Matter More Than N-grams (in this domain): For the JUPITER domain, the improvement came primarily from having multiple OOV networks (representing different classes), rather than just multiple OOV n-gram language models. This suggests that the phonetic structure and subword modeling within the OOV network are more critical than the contextual language modeling for this specific task.

Key Results

  • Significant False Alarm Reduction: For an OOV detection rate of 70%, the false alarm rate was reduced from 5.3% with a single OOV class to 2.9% with an eight-class POS model.
  • Improved Figure of Merit (FOM): The FOM, which measures performance in the low false alarm rate region, improved substantially. The baseline single-class model had an FOM of 0.64. The eight-class POS model achieved an FOM of 0.68, and the automatically derived multi-class model with POS initialization reached an FOM of 0.72.
  • Automatic Model Outperforms POS: The automatically derived multi-class model (using clustering) with POS initialization achieved the best performance (FOM of 0.72), outperforming both the single-class baseline and the POS-based multi-class model.
  • Number of Classes: For the automatic approach, most of the gain was achieved by going from one to two classes. Increasing the number of classes beyond eight provided diminishing returns in the JUPITER domain.

Limitations and Future Work

  • Domain Specificity: The authors note that the finding that OOV networks are more important than n-grams might be specific to the JUPITER domain, which is relatively simple and dominated by names and weather terms (nouns). They suggest that more unconstrained, large-vocabulary domains might benefit more from multiple OOV n-gram classes.
  • Contextual Similarity: While clustering groups phonetically similar words, they don't necessarily share similar contexts, which might limit further gains in some domains.
  • Future Work:
    • Combine multi-class approaches with multi-phone units within each OOV network.
  • Explore using this approach for detecting out-of-domain utterances.
  • Investigate using a second-stage search with a large off-line dictionary to identify the specific OOV words after detection.

Summary updated

BibTeX
@inproceedings{bazzi2002multi,
  author = {Issam Bazzi and James R. Glass},
  title = {{A multi-class approach for modelling out-of-vocabulary words}},
  booktitle = {Interspeech},
  year = {2002},
  doi = {10.21437/icslp.2002-480},
}
Figure