Probability-Aware Word-Confusion-Network-To-Text Alignment Approach for Intent Classification

Esaú Villatoro-Tello, Srikanth R. Madikeri, Bidisha Sharma, Driss Khalil, Shashi Kumar, Iuliia Nigmatulina, Petr Motlícek, Aravind GanapathirajuICASSP 2024 · OpenReview

Abstract

Spoken Language Understanding (SLU) technologies have greatly improved due to the effective pretraining of speech representations. A common requirement of industry-based solutions is the portability to deploy SLU models in voice-assistant devices. Thus, distilling knowledge from large text-based language models has become an attractive solution for achieving good performance and guaranteeing portability. In this paper, we introduce a novel architecture that uses a cross-modal attention mechanism to extract bin-level contextual embeddings from a word-confusion network (WNC) encoding such that these can be directly compared and aligned with traditional text-based contextual embeddings. This alignment is achieved using a recently proposed tokenwise constrastive loss function. We validate our architecture’s effectiveness by fine-tuning our WCN-based pretrained model to do intent classification (IC) on the well-known SLURP dataset. Obtained accuracy on the IC task (81%), depicts a 9.4% relative improvement compared to a recent/equivalent E2E method.
BibTeX
@inproceedings{villatorotello2024probability,
  author = {Esaú Villatoro-Tello and Srikanth R. Madikeri and Bidisha Sharma and Driss Khalil and Shashi Kumar and Iuliia Nigmatulina and Petr Motlícek and Aravind Ganapathiraju},
  title = {{Probability-Aware Word-Confusion-Network-To-Text Alignment Approach for Intent Classification}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2024},
  url = {https://openreview.net/forum?id=ZPCeBwExTc},
}

Loading related content...

Figure