Building Accurate Low Latency ASR for Streaming Voice Search in E-commerce

Abhinav Goyal, Nikesh GareraACL 2023

TL;DR

This paper tackles the challenge of building accurate, low-latency Automatic Speech Recognition (ASR) for streaming voice search in Hinglish (Hindi-English mix) for e-commerce. They introduce modifications to LSTM-CTC models, including windowed attention and hierarchical CTC loss, to improve accuracy while maintaining streaming capabilities. Crucially, they also propose a joint ASR and End-of-Speech (EOS) detection method within the CTC framework, significantly reducing perceived latency.

What Problem Does This Paper Address?

Voice search is becoming increasingly important, especially in markets like India where users may prefer voice interaction over typing. For e-commerce platforms, this means providing a seamless and responsive voice search experience. The core challenge lies in building ASR systems that are not only accurate but also low-latency and capable of streaming audio in real-time. Traditional ASR pipelines are often complex, and while end-to-end models like CTC, Attention-based Encoder-Decoder (AED), and RNN-Transducer (RNNT) have shown promise, adapting them for streaming while maintaining accuracy is difficult. Streaming models only have access to past audio, which can lead to reduced accuracy compared to models that can see the entire utterance. Furthermore, accurately detecting when a user has finished speaking (End-of-Speech or EOS) is critical for a good user experience, and relying on separate Voice Activity Detection (VAD) models can be suboptimal. This paper aims to bridge the accuracy gap between streaming and non-streaming ASR models and to integrate EOS detection directly into the ASR system for improved efficiency.

Methods

The authors build upon a Connectionist Temporal Classification (CTC) based ASR system, specifically an LSTM-attention architecture. Key modifications include:

  • Windowed Self-Attention: To improve the context captured by the LSTM, they incorporate multi-head self-attention but restrict its scope to a 5-frame window around the current time step. This allows the model to leverage local context without sacrificing streaming capabilities.
  • Hierarchical CTC (HCTC) Loss: Instead of a single CTC loss, they use a 3-level hierarchical structure that predicts characters, short subwords, and long subwords. This hierarchical loss is designed to implicitly relax the conditional independence assumption of CTC and improve transcription accuracy.
  • Label Smoothing: A standard technique to mitigate overconfidence in the model's output distributions.
  • Joint ASR and EOS Detection: A novel approach is proposed to integrate EOS detection directly into the CTC model. This involves appending a special < /s > token to the vocabulary and training the model with early/late penalties to encourage it to predict this token at the end of speech. During inference, a dynamic threshold based on the probability of this token is used to determine the EOS.

Datasets

The primary dataset consists of approximately 14 million audio-text pairs from Flipkart's e-commerce voice search queries, supplemented with data from customer support calls and crowd-sourced read speech. For evaluation, about 19,000 audio samples from e-commerce voice search queries were manually transcribed and verified. The test set is further divided into "clean" and "noisy" subsets.

Key Insights and Ideas

  • Bridging the Streaming Gap: The combination of windowed self-attention and hierarchical CTC loss significantly improves the accuracy of streaming ASR models, bringing them closer to the performance of non-streaming models.
  • Integrated EOS Detection is Key: Jointly training for ASR and EOS detection within the CTC framework is more efficient and effective than using separate VAD models. This integration directly addresses user-perceived latency.
  • Hierarchical Structure Benefits: The multi-level prediction (characters, short/long subwords) in HCTC helps the model learn a richer representation of speech, leading to better transcription.
  • Local Context Matters: Windowed attention, while limiting the scope, effectively captures local dependencies crucial for accurate streaming transcription without the computational burden of full attention.

Key Results

  • The proposed LSTM-attention HCTC model with joint EOS detection achieves a Word Error Rate (WER) of 3.69% (without EOS) and 4.78% (with EOS) on their e-commerce Hinglish voice search data.
  • The integrated EOS detection significantly reduces search latency by approximately 1300 ms (a 46.64% reduction) compared to using an independent VAD model.
  • The model shows substantial performance gains over a vanilla LSTM-CTC model, with a relative improvement of around 30% in WER.
  • The proposed model performs comparably to a streaming Conformer-CTC model but with lower latency.

Limitations and Future Work

  • Data Size: The study was conducted in a high-resource setting with about 22.5k hours of labeled data. The authors acknowledge that for languages with less data, their approach might need further adaptation.
  • Model Size: They employed a smaller model (around 60 million parameters) to accommodate deployment constraints, suggesting that larger models could potentially yield even better performance.
  • RNNT Models: RNNT models were not included in the comparison due to their higher computational requirements.
  • Future Directions: The authors plan to explore leveraging large unsupervised datasets, larger models, and extending their work to other Indian languages. They are also motivated to develop a single joint model for ASR, EOS detection, and Spoken Language Understanding (SLU), and to explore multilingual ASR models.

Summary updated

BibTeX
@inproceedings{goyal2023building,
  author = {Abhinav Goyal and Nikesh Garera},
  title = {{Building Accurate Low Latency ASR for Streaming Voice Search in E-commerce}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure