Improving Pseudo-label Training For End-to-end Speech Recognition Using Gradient Mask
Lists
Only you see this
Abstract
In the recent trend of semi-supervised speech recognition, both self-supervised representation learning and pseudo-labeling have shown promising results. In this paper, we propose a novel approach to combine their ideas for end-to-end speech recognition model. Without any extra loss function, we utilize the Gradient Mask to optimize the model when training on pseudo-label. This method forces the speech recognition model to predict from the masked input to learn strong acoustic representation and make training robust to label noise. In our semi-supervised experiments, the method can improve the model performance when training on pseudo-label and our method achieved competitive results comparing with other semi-supervised approaches on the Librispeech 100 hours experiments.
BibTeX
@inproceedings{ling2022improving,
author = {Shaoshi Ling and Chen Shen and Meng Cai and Zejun Ma},
title = {{Improving Pseudo-label Training For End-to-end Speech Recognition Using Gradient Mask}},
booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
year = {2022},
doi = {10.1109/icassp43922.2022.9746249},
}Loading related content...