Improvements in Japanese voice search

Ken-ichi Iso, Edward Whittaker, Tadashi Emori, Junpei MiyakeInterspeech 2012

TL;DR

This paper presents significant improvements to a Japanese voice search system, focusing on memory efficiency and language model adaptation. By compressing the internal weighted finite-state transducer (WFST) representation and developing a novel method for integrating manually transcribed spoken queries, the researchers achieved a more accurate and resource-efficient voice search experience.

What Problem Does This Paper Address?

Voice search, particularly for languages like Japanese, presents unique challenges. These include the need to handle large vocabularies to minimize out-of-vocabulary (OOV) rates, the complexity of Japanese text segmentation and multiple orthographies, and the substantial memory requirements of traditional speech recognition decoders. Existing open-source decoders, while functional, often struggle with these demands, especially in resource-constrained environments like mobile devices. This paper aims to address these limitations by developing a more efficient internal decoder and improving language model adaptation techniques for Japanese voice search.

Methods

The core of the paper lies in two main areas: decoder improvements and language model adaptation.

Decoder Improvements: * Compressed WFST Arc Representation: The authors developed a novel way to represent WFST arcs by exploiting the limited number of unique input/output label combinations. Instead of storing full indices, they use a 3-byte index for the input/output label pair and a 1-byte index for quantized weights, reducing the memory footprint per arc. * Arc Weight Quantization: Arc weights were quantized to 8 bits without impacting performance, further reducing memory usage. * Standard WFST Construction: The recognition network was built using a standard WFST composition pipeline involving context-dependency transducers, lexicon, and language models, leveraging tools like Juicer and OpenFST.

Language Model Adaptation: * WFST-based Segmentation Alignment: A key innovation is a sequence of WFST operations designed to align manually transcribed spoken queries with the system's internal segmentation. This process ensures consistency between human annotations and the decoder's output, which is crucial for effective adaptation. * Interpolation-based LM Adaptation: The baseline language model, trained on written queries, is interpolated with a new language model trained on manually transcribed spoken queries. This allows the system to adapt to the style and distribution of spoken queries.

Datasets

  • Training Data:
    • 1.7 billion unique written queries (segmented into word-like units) for language model training.
  • 500 hours of anonymized voice queries (collected March 2011 - September 2011) for acoustic model training.
  • Development Data:
    • 10,000 anonymized spoken utterances (collected September 2011) for parameter optimization.
  • Evaluation Data:
    • 10,000 anonymized spoken utterances (collected January 2012) for final performance evaluation.
  • LM Adaptation Data:
    • 1.6 million spoken transcriptions (collected March 2011 - September 2011) used for language model adaptation experiments.

Key Insights and Ideas

  • Memory Efficiency is Crucial: The paper strongly demonstrates that optimizing memory usage in WFST-based decoders is paramount, especially for large-vocabulary tasks like voice search. Their compressed representation significantly reduces memory overhead without sacrificing accuracy.
  • Bridging the Spoken-Written Gap: Manually transcribing spoken queries is labor-intensive. The WFST-based alignment method is a clever way to leverage these valuable annotations by ensuring they are compatible with the existing system's structure, enabling effective language model adaptation.
  • Simple Adaptation Yields Significant Gains: Even a straightforward interpolation-based LM adaptation using a relatively small set of transcribed spoken queries (up to 1 million) led to noticeable improvements in sentence and character error rates. This highlights the power of adapting to the target domain.
  • Open-Source vs. Internal Decoders: While open-source decoders like Julius and Juicer are valuable, the paper shows that a carefully optimized internal decoder can outperform them in terms of both accuracy and resource efficiency for specific tasks.

Key Results

  • Memory Reduction: The internal decoder uses 2.3GB of memory, which is 76% less than the comparable open-source decoder (Juicer at 9.9GB).
  • Performance Comparison: On the evaluation set, the internal decoder achieved better accuracy (lower Sentence Error Rate - SER) at comparable real-time factors (RTF) compared to Julius and Juicer.
  • LM Adaptation Gains:
    • Relative reduction of up to 4.6% in Sentence Error Rate (SER).
  • Relative reduction of up to 8.2% in Character Error Rate (CER) when adapting with 1 million spoken transcriptions.

Limitations and Future Work

  • Manual Transcription Dependency: The current LM adaptation relies on manually transcribed spoken queries. The authors suggest future work to explore more effective language model adaptation using automatic transcriptions.
  • Further Adaptation Techniques: Future work will also focus on exploring more advanced language model adaptation techniques beyond simple interpolation.
  • WFST Compression Complexity: While the chosen compression method offers a good balance between performance and complexity, the paper acknowledges that alternative implementations might achieve greater memory reductions at the cost of increased complexity.

Summary updated

BibTeX
@inproceedings{iso2012improvements,
  author = {Ken-ichi Iso and Edward Whittaker and Tadashi Emori and Junpei Miyake},
  title = {{Improvements in Japanese voice search}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure