When Hearing the Voice, Who Will Come to Your Mind
Lists
Only you see this
Abstract
Speech is a carrier containing rich biological information, such as speaker identity information including age, gender, race. In this paper, we explore the use of a self-supervised method to obtain speaker identity information from high-dimensional speech representations to generate face image. At the same time, considering that the biological information contained in the same piece of speech has different expression forms (such as images), we designed a cross-modal knowledge distillation method to transform the feature information from the visual domain to the speech domain. The feature vectors obtained through self-supervised learning and knowledge distillation are fed into a GAN-based generative model to obtain facial images containing speaker information. Subjective experiments show that our model can reach a well performance in the task of speaker identification. Experiments show that our proposed method can effectively establish the connection between different modalities and generate a face with rich biological information.
BibTeX
@inproceedings{hong2021when,
author = {Zhenhou Hong and Jianzong Wang and Wenqi Wei and Jie Liu and Xiaoyang Qu and Bo Chen and Zihang Wei and Jing Xiao},
title = {{When Hearing the Voice, Who Will Come to Your Mind}},
booktitle = {IJCNN},
year = {2021},
doi = {10.1109/ijcnn52387.2021.9534208},
}Loading related content...