OOV Word Detection using Hybrid Models with Mixed Types of Fragments

Long Qin, Alexander I. RudnickyInterspeech 2012 · OpenReview

Abstract

This paper presents initial studies to improve the out-of-vocabulary (OOV) word detection performance by using mixed types of fragment units in one hybrid system. Three types of fragment units—subwords, syllables, and graphones—were combined in two different ways to build the hybrid lexicon and language model. Experimental results demonstrate that hybrid systems with mixed types of fragment units perform better than systems using only one type. Based on an analysis of unit number and length, future work is proposed to optimize the utilization of mixed fragment types.

Introduction

Automatic Speech Recognition (ASR) systems typically operate with a "closed vocabulary," meaning they can only recognize words explicitly defined in their lexicon. When a user utters an Out-of-Vocabulary (OOV) word—such as a proper noun or technical term—the system often misidentifies it as the phonetically closest in-vocabulary word. This leads to recognition errors and prevents the system from flagging unknown terms for further processing.

To address this, researchers employ "hybrid models" that represent OOV words as sequences of sub-lexical fragments. However, different fragment types have distinct trade-offs:

  • Subwords (phone sequences) are robust but lack linguistic structure.
  • Syllables follow phonological rules but can be rare and difficult to train.
  • Graphones (joint letter-sound units) capture orthographic and phonetic information but often result in short units that increase recognition ambiguity.

This study explores whether combining these fragment types can leverage their individual strengths while mitigating their weaknesses.

Methodology: Mixed-Unit Hybrid Systems

The researchers developed a hybrid lexicon and language model (LM) where OOV words are replaced by fragment sequences. They introduced two primary strategies for mixing units:

  1. Hierarchical Mixing (mix-hier): OOV words are ranked by frequency. The most frequent are modeled by syllables, the next tier by graphones, and the remainder by subwords. This approach requires manual tuning of frequency thresholds.
  2. Stochastic Mixing (mix-flat): For each OOV occurrence in the training data, the system randomly selects one of the three fragment types. This provides broader coverage and eliminates the need for manual threshold tuning.

To maintain word boundaries during recognition, special "word-start" ($\wedge$) and "word-end" ($\$$) symbols were utilized.

Experimental Setup

The systems were evaluated on two standard ASR benchmarks:

  • Broadcast News (BN): Using HUB4-96 data.
  • Wall Street Journal (WSJ): Using WSJ0 and BN 92-96 text corpora.

In both tasks, the vocabulary was limited to the 20,000 most frequent words to maintain a consistent OOV rate of approximately 2% for testing.

Results and Discussion

The experimental results consistently showed that mixed-unit systems outperform single-unit baselines.

Broadcast News Task

In the BN task, the mix-flat system demonstrated superior performance across both development and evaluation sets. Syllables were the strongest single-unit performer in this domain, likely due to their longer average length providing more acoustic stability.

Wall Street Journal Task

Similar trends were observed in the WSJ task. The mix-flat approach again provided the best OOV detection performance. Analysis of error distributions revealed that over 50% of detection errors occurred in only one of the three single-unit systems. By combining them, the hybrid model effectively "smoothed out" these isolated failures.

Performance comparison on the Wall Street Journal (WSJ) task for both Dev and Eval sets.

Key Insights

  • Complementarity: Only 22-23% of errors were shared across all fragment types, confirming that different units fail in different ways.
  • Unit Length: Longer fragment units generally performed better as they provide more acoustic context for the recognizer.
  • Stochastic Effectiveness: The mix-flat method proved highly effective and easier to implement than mix-hier, as it does not require a development set for threshold optimization.

Conclusion and Future Work

The study confirms that using mixed types of fragment units significantly improves OOV word detection. While the stochastic mix-flat system is robust, future work aims to develop a more principled assignment strategy: using syllables as a base, breaking rare syllables into subwords to prevent overfitting, and employing graphones for short, frequent units to leverage orthographic information. Additionally, increasing the sequence length of graphones may further enhance their competitiveness.

Summary updated

Abstract

This paper presents initial studies to improve the out-of-vocabulary (OOV) word detection performance by using mixed types of fragment units in one hybrid system. Three types of fragment units, subwords, syllables, and graphones, were combined in two different ways to build the hybrid lexicon and language model. The experimental results show that hybrid systems with mixed types of fragment units perform better than hybrid systems using only one type of fragment unit. After comparing the OOV word detection performance with the number and length of fragment units of each system, we proposed future work to better utilize mixed types of fragment units in a hybrid system.
BibTeX
@inproceedings{qin2012oov,
  author = {Long Qin and Alexander I. Rudnicky},
  title = {{OOV Word Detection using Hybrid Models with Mixed Types of Fragments}},
  booktitle = {Interspeech},
  year = {2012},
  url = {https://openreview.net/forum?id=4370slz3ox_},
}
Figure