Towards High-Performance and Low-Latency Feature-Based Speaker Adaptation of Conformer Speech Recognition Systems
Lists
Only you see this
Abstract
Practical application of model-based speaker adaptation techniques to end-to-end ASR systems is hindered by speaker-level data scarcity and latency in speaker-dependent (SD) parameters update. To this end, data-efficient and low-latency rapid feature-based speaker adaptation approaches are proposed in this paper for state-of-the-art Conformer ASR systems. Compact subspace projection of training data estimated SD hidden layer output scaling or bias parameters is used to represent the most distinctive speaker "bases". A feature-driven prediction network containing purpose-built speaker-aware memory is designed to on-the-fly produce homogeneous SD basis interpolation, and facilitate rapid speaker adaptation. Experimental results on the 300-hr Switchboard corpus suggest that the proposed adaptation approach produces statistically significant word error rate (WER) reductions of up to 1.0% absolute (8.4% relative) over the baseline speaker-independent and i-vector adapted Conformers before and after external LM rescoring. Consistent WER reductions of up to 2.0% absolute (16.3% relative) and real-time factor speeding up ratios of up to 10.9 times are also obtained over offline model-based adaptation across different speaker-level data quantity operating points. T-SNE visualization reveals the on-the-fly predicted SD basis weights present intuitively more consistent speaker features than i-vectors.
BibTeX
@inproceedings{deng2024towards,
author = {Jiajun Deng and Xurong Xie and Guinan Li and Mingyu Cui and Mengzhe Geng and Zengrui Jin and Tianzi Wang and Shujie Hu and Zhaoqing Li and Xunying Liu},
title = {{Towards High-Performance and Low-Latency Feature-Based Speaker Adaptation of Conformer Speech Recognition Systems}},
booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
year = {2024},
doi = {10.1109/icassp48485.2024.10448488},
}Loading related content...