A transfer learning and progressive stacking approach to reducing deep model sizes with an application to speech enhancement
Lists
Only you see this
Abstract
Leveraging upon transfer learning, we distill the knowledge in a conventional wide and deep neural network (DNN) into a narrower yet deeper model with fewer parameters and comparable system performance for speech enhancement. We present three transfer-learning solutions to accomplish our goal. First, the knowledge embedded in the form of the output values of a high-performance DNN is used to guide the training of a smaller DNN model in sequential transfer learning. In the second multi-task transfer learning solution, the smaller DNN is trained to learn the output value of the larger DNN, and the speech enhancement task in parallel. Finally, a progressive stacking transfer learning is accomplished through multi-task learning, and DNN stacking. Our experimental evidences demonstrate 5 times parameter reduction while maintaining similar enhancement performance with the proposed framework.
BibTeX
@inproceedings{wang2017transfer,
author = {Sicheng Wang and Kehuang Li and Zhen Huang and Sabato Marco Siniscalchi and Chin-Hui Lee},
title = {{A transfer learning and progressive stacking approach to reducing deep model sizes with an application to speech enhancement}},
booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
year = {2017},
doi = {10.1109/icassp.2017.7953223},
}Loading related content...