Deep Grammatical Multi-classifier for Continuous Sign Language Recognition
Lists
Only you see this
Abstract
In this paper, we propose a novel deep architecture with multiple classifiers for continuous sign language recognition. Representing the sign video with a 3D convolutional residual network and a bidirectional LSTM, we formulate continuous sign language recognition as a grammatical-rule-based classification problem. We first split a text sentence of sign language into isolated words and n-grams, where an n-gram is a sequence of consecutive n words in a sentence. Then, we propose a word-independent classifiers (WIC) module and an n-gram classifier (NGC) module to identify the words and n-grams in a sentence, respectively. A greedy decoding algorithm is employed to integrate words and n-grams into the sentence based on the confidence scores provided by both modules. Our method is evaluated on a Chinese continuous sign language recognition benchmark, and the experimental results demonstrate its effectiveness and superiority.
BibTeX
@inproceedings{wei2019deep,
author = {Chengcheng Wei and Wengang Zhou and Junfu Pu and Houqiang Li},
title = {{Deep Grammatical Multi-classifier for Continuous Sign Language Recognition}},
booktitle = {BigMM},
year = {2019},
doi = {10.1109/bigmm.2019.00027},
}Loading related content...