Similarity scoring for recognizing repeated out-of-vocabulary words

Mirko Hannemann, Stefan Kombrink, Martin Karafiát, Lukáš BurgetInterspeech 2010

TL;DR

This paper tackles the challenge of identifying and clustering repeated Out-of-Vocabulary (OOV) words in speech recognition. They introduce a novel similarity measure based on an alignment error model that leverages sub-word sequences from hybrid recognizers, enabling more accurate OOV detection and description than traditional methods. This work is crucial for improving information retrieval and understanding in speech systems by capturing important, yet often missed, domain-specific terms.

What Problem Does This Paper Address?

Speech recognition systems typically have a fixed vocabulary. When encountering words not in this vocabulary (OOVs), they often replace them with similar-sounding in-vocabulary words, leading to a loss of crucial information, especially for topic-specific terms. While some OOV detection methods exist, they struggle with partial detections and accurately identifying the start and end of OOVs. This paper addresses the need for a robust way to not only detect OOVs but also to identify when the same OOV appears multiple times within a speech segment. This is vital for tasks like information retrieval, where recognizing recurring keywords is essential for accurate indexing and search.

Methods

The authors employ a hybrid word/sub-word recognizer that can output sequences of sub-words (like phones or syllables) when it encounters words it doesn't recognize. These sub-word sequences serve as "OOV candidates" and provide a more precise boundary for the detected OOV than traditional methods.

The core of their contribution is a similarity measure designed to compare these OOV candidates. This measure is based on an alignment error model. Instead of a direct string comparison, they align the sub-word sequences of two OOV candidates. This alignment process allows for:

  • Deletions, substitutions, and insertions: Accounting for errors in the recognized sub-word sequences.
  • Partial overlap: Recognizing that an OOV might not be perfectly detected each time.
  • Stripping/adding prefixes/suffixes: Handling variations where parts of the OOV might be recognized or missed.

This alignment process results in a lattice of possible alignments, each with a score representing the cost of that alignment. They then use these scores to determine similarity. For "query-by-example," they find other similar OOV candidates or in-vocabulary words. For "higher-level OOV description," they aim to identify relationships like compound words or derivations.

Datasets

  • Fisher database: Used for training and evaluation. Specifically, 57 hours of noisy, conversational speech were used as a test set, with a reduced vocabulary to artificially increase the OOV rate.
  • TED database: Mentioned as a future application for testing with a non-artificially limited vocabulary.
  • Switchboard data: Used for training acoustic models and word language models.
  • Hub5 Eval01: Used as development data to train the error model.

Key Insights and Ideas

  • Sub-word sequences are powerful OOV descriptors: The hybrid recognizer's ability to output sub-word sequences provides a richer representation of OOVs than simple word replacements, enabling more precise detection and comparison.
  • Alignment is key to handling OOV variability: Recognizing that OOVs are rarely detected perfectly, the alignment error model is crucial for comparing potentially noisy and incomplete sub-word sequences. This allows for identifying related words even with significant variations.
  • OOVs often have linguistic relationships: The paper highlights that many OOVs are not entirely novel but are related to known words through derivation (e.g., "abnormalities" from "abnormal") or compounding (e.g., "counterilluminate" from "counter" and "illuminate"). Their method aims to capture these relationships.
  • The challenge of ambiguity: Even with sub-word sequences, a single OOV candidate might correspond to multiple reference words (e.g., "kaort ehks" for "cortex," "neocortex," and "Gore-Tex"), indicating the inherent ambiguity in speech recognition.

Key Results

  • Improved OOV Detection: Table 1 shows that their hybrid word/sub-word recognizer achieves a reasonable OOV precision (85.3%) and recall (23.3%) on the Fisher data with a reduced vocabulary.
  • Clustering Performance: Table 2 demonstrates that their similarity scoring and clustering approach, using different beamwidths for the alignment lattice, yields moderate results. For instance, with a beamwidth of 6.0, they achieve around 60% recall for reoccurring OOVs and cluster accuracies and completeness in the 40-60% range.
  • OOV Description Accuracy: The accuracy of describing OOVs as compounds or derivations is also in the 40-50% range, indicating potential for extracting higher-level linguistic information.

Limitations and Future Work

  • Limitations of One-Best Output: The current work primarily uses the one-best output from the ASR. The authors acknowledge that using ASR lattices and more sophisticated probabilistic or fuzzy clustering algorithms could improve performance.
  • Artificially High OOV Rate: The use of a reduced vocabulary in the Fisher dataset artificially inflated the OOV rate, which might not reflect real-world scenarios with larger vocabularies.
  • Future Work:
    • Applying the techniques to the TED database with a larger, non-artificially limited vocabulary.
  • Switching from one-best ASR output to ASR lattices.
  • Implementing more sophisticated clustering algorithms (probabilistic or fuzzy).
  • Training the error model on lattices or using query-by-example/keyword spotting on lattices.
  • Developing a more sophisticated layered error model to better handle compounding and derivation costs.

Summary updated

BibTeX
@inproceedings{hannemann2010similarity,
  author = {Mirko Hannemann and Stefan Kombrink and Martin Karafiát and Lukáš Burget},
  title = {{Similarity scoring for recognizing repeated out-of-vocabulary words}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure