Multimodal Face-Pose Estimation With Multitask Manifold Deep Learning
Lists
Only you see this
Abstract
Face-pose estimation aims at estimating the gazing direction with two-dimensional face images. It gives important communicative information and visual saliency. However, it is challenging because of lights, background, face orientations, and appearance visibility. Therefore, a descriptive representation of face images and mapping it to poses are critical. In this paper, we use multimodal data and propose a novel face-pose estimation framework named multitask manifold deep learning ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{M}^2\text{DL}$</tex-math></inline-formula> ). It is based on feature extraction with improved convolutional neural networks (CNNs) and multimodal mapping relationship with multitask learning. In the proposed CNNs, manifold regularized convolutional layers learn the relationship between outputs of neurons in a low-rank space. Besides, in the proposed mapping relationship learning method, different modals of face representations are naturally combined by applying multitask learning with incoherent sparse and low-rank learning with a least-squares loss. Experimental results on three challenging benchmark datasets demonstrate the performance of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{M}^2\text{DL}$</tex-math></inline-formula> .
BibTeX
@article{hong2019multimodal,
author = {Chaoqun Hong and Jun Yu and Jian Zhang and Xiongnan Jin and Kyong-Ho Lee},
title = {{Multimodal Face-Pose Estimation With Multitask Manifold Deep Learning}},
journal = {IEEE Trans. Industrial Informatics},
year = {2019},
doi = {10.1109/tii.2018.2884211},
}Loading related content...