Vocabulary independent spoken query: a case for subword units

Evandro Gouvêa, Tony EzzatInterspeech 2010

TL;DR

This paper introduces a novel approach to voice-based information retrieval (IR) that uses subword units instead of full words. This "vocabulary-independent" system significantly improves robustness to out-of-vocabulary (OOV) words, a common problem in speech recognition, by decoupling the Automatic Speech Recognition (ASR) engine from the IR task.

What Problem Does This Paper Address?

Voice-based information retrieval is becoming increasingly important, especially for complex queries on mobile devices. A typical system involves an ASR front-end that converts spoken queries into word hypotheses, which are then fed to an IR back-end. A major challenge is that the ASR's language model (LM) is often built from the same data as the IR index. This means that if the indexed data changes, the LM must be rebuilt, which is computationally expensive and time-consuming. Furthermore, new or rare words not present in the original training data lead to "out-of-vocabulary" (OOV) errors, severely degrading retrieval performance. This paper tackles the problem of creating robust voice search systems that are independent of the indexed data and resilient to OOV words.

Methods

The core innovation is the use of subword units as the fundamental building blocks for both the ASR language model and the IR index. Instead of full words, the system breaks down pronunciations into smaller phonetic units.

  • MDL Subword Unit Inventory: The paper employs an algorithm inspired by the Minimum Description Length (MDL) principle to create a subword unit inventory. This algorithm aims to find a set of subword units that minimizes the combined cost of representing the training data and storing the inventory itself.
  • Vocabulary Independence: By using subword units, the ASR engine's language model and pronunciation dictionary become independent of the specific words in the IR database. When the database changes, only the IR index needs to be rewritten in terms of the existing subword units, not the entire ASR model.
  • Rewriting Database and LM: A Viterbi algorithm is used to segment pronunciations into subword units from the learned inventory. This allows for efficient rewriting of the database and training of the LM on subword units.

Datasets

  • Song Lyrics Dataset: The experiments were conducted on a dataset of 35,868 songs, including titles, artists, albums, and lyrics. This dataset was used to simulate a "song lyric search" task where users speak portions of lyrics.
  • Subsets for Training and Evaluation: Two subsets of the song database were used:
  • ls2000 (1989 songs): Used for building the subword unit inventory and for training the ASR language model.
  • ls36000 (all songs): Used for evaluating retrieval performance.
  • Simulated OOV Rates: The paper simulates OOV rates by pruning the dictionary and language model, creating scenarios with 5%, 20%, 40%, 60%, and 80% OOV words.

Key Insights and Ideas

  • Decoupling ASR and IR: The most significant insight is that by using subword units, the ASR system can remain fixed while the IR index is updated. This drastically reduces the computational cost and complexity of maintaining voice search systems for dynamic datasets.
  • Robustness to OOV: Subword units are inherently more robust to OOV words than word-based systems. When a word is not in the vocabulary, a word-based system fails, whereas a subword system can still represent it using its constituent subword units.
  • Subword Units Generalize Better: The paper suggests that subword unit inventories built from smaller, more constrained datasets can generalize better to unseen data compared to those built from larger, more diverse datasets.
  • Inventory Size Matters, but Not Excessively: While the recall performance is sensitive to the subword unit inventory size (larger inventories generally perform better), the degradation is much more graceful than the catastrophic failure seen in word-based systems as OOV rates increase.

Key Results

  • Superior Recall at High OOV Rates: For a music lyrics task with an 80% OOV rate, the subword-based system achieved a recall of 75.2%, significantly outperforming the word-based system's 47.4%.
  • Graceful Degradation of Phone Error Rate (PER): Figure 3 shows that subword systems exhibit a much more graceful degradation in PER as OOV rates increase compared to word-based systems.
  • Consistent Recall Performance: Figure 5 demonstrates that the subword system maintains a reasonable recall level even at high OOV rates, while the word system's recall plummets.
  • Impact of Inventory Size: Figure 4 shows that recall performance is significantly affected by subword unit inventory size, with larger inventories (e.g., >1000 units) generally yielding better results.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that the assumption of a fixed ASR LM and a dynamic pronunciation dictionary is reasonable for embedded systems but might not apply universally.
  • The study focused on a specific task (music lyrics search) and dataset.
  • Future Work:
    • The authors plan to investigate the generality of their findings by applying the algorithms to other ASR and IR platforms.
  • They also aim to explore the application of subword units to different types of datasets beyond music lyrics.

Summary updated

BibTeX
@inproceedings{gouvea2010vocabulary,
  author = {Evandro Gouvêa and Tony Ezzat},
  title = {{Vocabulary independent spoken query: a case for subword units}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure