Spectral learning of latent semantics for action recognition

Zhiwu Lu, Yuxin Peng, Horace Ho-Shing Ip

Abstract

This paper proposes novel spectral methods for learning latent semantics (i.e. high-level features) from a large vocabulary of abundant mid-level features (i.e. visual keywords), which can help to bridge the semantic gap in the challenging task of action recognition. To discover the manifold structure hidden among mid-level features, we develop spectral embedding approaches based on graphs and hypergraphs, without the need to tune any parameter for graph construction which is a key step of manifold learning. In particular, the traditional graphs are constructed by linear reconstruction with sparse coding. In the new embedding space, we learn high-level latent semantics automatically from abundant mid-level features through spectral clustering. The learnt latent semantics can be readily used for action recognition with SVM by defining a histogram intersection kernel. Different from the traditional latent semantic analysis based on topic models, our two spectral methods for semantic learning can discover the manifold structure hidden among mid-level features, which results in compact but discriminative high-level features. The experimental results on two standard action datasets have shown the superior performance of our spectral methods.
BibTeX
@inproceedings{lu2011spectral,
  author = {Zhiwu Lu and Yuxin Peng and Horace Ho-Shing Ip},
  title = {{Spectral learning of latent semantics for action recognition}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2011},
  doi = {10.1109/iccv.2011.6126408},
}

Loading related content...

Figure