Combination of Pyramid CNN Representation and Spatial-Temporal Representation for Facial Expression Recognition

Shulin Xu, Nan Pu, Li Qian, Guoqiang Xiao

Abstract

In this paper, we propose a novel framework for facial expression recognition in video sequences by combining deep convolutional feature and spatial-temporal feature. Firstly, apex frame of every sequence is selected adaptively by calculating the displacement of facial landmarks. Then, pyramid CNN-based feature is extracted on the apex frame to capture the information of global and local regions of human face. Afterwards, spatial-temporal LBP-TOP feature is generated from video sequence and integrated with pyramid CNN-based feature to represent video, which reflect dynamic and static texture information of facial expressions. Finally, the multiclass support vector machine (SVM) with one-versus-one strategy is applied to classify facial expressions. Experimental results on the extended Cohn-Kanade (CK+) and Oulu-CASIA datasets demonstrate the superiority of our proposed method.
BibTeX
@inproceedings{xu2017combination,
  author = {Shulin Xu and Nan Pu and Li Qian and Guoqiang Xiao},
  title = {{Combination of Pyramid CNN Representation and Spatial-Temporal Representation for Facial Expression Recognition}},
  booktitle = {CCCV},
  year = {2017},
  doi = {10.1007/978-981-10-7302-1\_4},
}

Loading related content...

Figure