Meta-Adapter: Efficient Cross-Lingual Adaptation With Meta-Learning

Wenxin Hou, Yidong Wang, Shengzhou Gao, Takahiro Shinozaki

Abstract

Transfer learning from a multilingual model has shown favorable results on low-resource automatic speech recognition (ASR). However, full-model fine-tuning generates a separate model for every target language and is not suitable for deploying and maintaining in production. The key challenge lies in how to efficiently extend the pre-trained model with fewer parameters. In this paper, we propose to combine the adapter module with meta-learning algorithms to achieve high recognition performance under low-resource settings and improve the parameter-efficiency of the model. Extensive experiments show that our methods can achieve comparable or even superior recognition rates than the state-of-the-art baselines on low-resource languages, especially under very-low-resource conditions, with a significantly smaller model profile.
BibTeX
@inproceedings{hou2021meta,
  author = {Wenxin Hou and Yidong Wang and Shengzhou Gao and Takahiro Shinozaki},
  title = {{Meta-Adapter: Efficient Cross-Lingual Adaptation With Meta-Learning}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2021},
  doi = {10.1109/icassp39728.2021.9414959},
}

Loading related content...

Figure