Vararray Meets T-Sot: Advancing the State of the Art of Streaming Distant Conversational Speech Recognition
Lists
Only you see this
Abstract
This paper presents a novel streaming automatic speech recognition (ASR) framework for multi-talker overlapping speech captured by a distant microphone array with an arbitrary geometry. Our framework, named t-SOT-VA, capitalizes on independently developed two recent technologies; array-geometry-agnostic continuous speech separation, or VarArray, and streaming multi-talker ASR based on token-level serialized output training (t-SOT). To combine the best of both technologies, we newly design a t-SOT-based ASR model that generates a serialized multi-talker transcription based on two separated speech signals from VarArray. We also propose a pre-training scheme for such an ASR model where we simulate VarArray’s output signals based on monaural single-talker ASR training data. Conversation transcription experiments using the AMI meeting corpus show that the system based on the proposed framework significantly outperforms conventional ones. Our system achieves the state-of-the-art word error rates of 13.7% and 15.5% for the AMI development and evaluation sets, respectively, in the multiple-distant-microphone setting while retaining the streaming inference capability.
BibTeX
@inproceedings{kanda2023vararray,
author = {Naoyuki Kanda and Jian Wu and Xiaofei Wang and Zhuo Chen and Jinyu Li and Takuya Yoshioka},
title = {{Vararray Meets T-Sot: Advancing the State of the Art of Streaming Distant Conversational Speech Recognition}},
booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
year = {2023},
doi = {10.1109/icassp49357.2023.10095367},
}