Regression based landmark estimation and multi-feature fusion for visual speech recognition
Lists
Only you see this
Abstract
Visual speech recognition also known as lipreading can improve robustness of automatic acoustic speech recognition especially under noisy environments. However, it remains a challenging topic considering the variety of speaking characteristics and confusion between visual speech features. In this paper, we propose an automatic lipreading method by using a new lip tracking method and multiple visual information fusion to tackle the problem. First, a method of face landmark estimation based on regression is employed for lip detection, based on which a geometric-based shape invariant feature (SIF) is put forward. Moreover, it can also be applied to the removal of the non-speaking utterance. Then the motion interchange patterns and spatial-temporal descriptors are also adopted to describe the lip information, where the Bayes combination strategy is applied. The proposed method is explored on three benchmark data sets: Avletters2, OuluVS and PKUVS. Experimental results demonstrate promising results and show effectiveness of the proposed approach.
BibTeX
@inproceedings{liu2015regression,
author = {Hong Liu and Xuewu Zhang and Pingping Wu},
title = {{Regression based landmark estimation and multi-feature fusion for visual speech recognition}},
booktitle = {ICIP},
year = {2015},
url = {https://openreview.net/forum?id=DsB3VtSKsC},
}Loading related content...